2024-03-21T08:35:35.766 INFO:root:teuthology version: 0.0.1.dev262+ge691533 2024-03-21T08:35:35.766 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_vshankar@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} --name vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi --archive /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145 -- /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/orig.config.yaml 2024-03-21T08:35:35.819 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T08:35:35.924 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145 branch: wip-vshankar-testing-20240320.053002 description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} email: vshankar@redhat.com first_in_suite: false job_id: '7612145' kernel: client: sha1: distro flavor: default kdb: true sha1: 5ec3e6e88b2e2783e09498588bb1f433ae9c8986 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing-20240320.053002 ceph: cephfs: max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported dry_run: true install: ceph: flavor: default sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing-20240320.053002 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 owner: scheduled_vshankar@teuthology priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 8073 sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 sleep_before_teardown: 0 subset: 203/1024 suite: fs suite_branch: wip-vshankar-testing-20240320.053002 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi087.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfQ5mJ3bWnXAMBSktocr6hXpZC+vE9PwUKFZLqI7jI+ew4wzW7MsSXLDX2ELpAhsDehpUg+emWycV7YwX/Zhko= smithi100.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVVoaOBkE3CItZzR+zdxw5fQRlolyR8GPnQhe1YvWnJGeWD09s1hxpKJTGnOuvZmQDkItj4YHrSLawE8hZrMVE= smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4zlef7E7np1ld4jsMgNcjLRPZ+Q3Tck4iGeYrvwXb0VawcGgLDDTeIUkYSQnqH27s6fqHa2Ijd5Utipw+2dZA= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: e691533f9cbb33d85b2187bba20d7102f098636d timestamp: 2024-03-20_11:21:18 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-21T08:35:35.924 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa; will attempt to use it 2024-03-21T08:35:35.925 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_2edfee68ad7243bed422a049b6f5f58619300f11/qa/tasks 2024-03-21T08:35:35.925 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-21T08:35:35.926 INFO:teuthology.task.internal:Checking packages... 2024-03-21T08:35:35.950 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '2edfee68ad7243bed422a049b6f5f58619300f11' 2024-03-21T08:35:35.950 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-21T08:35:35.950 INFO:teuthology.packaging:ref: None 2024-03-21T08:35:35.950 INFO:teuthology.packaging:tag: None 2024-03-21T08:35:35.950 INFO:teuthology.packaging:branch: wip-vshankar-testing-20240320.053002 2024-03-21T08:35:35.950 INFO:teuthology.packaging:sha1: 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:35:35.950 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing-20240320.053002 2024-03-21T08:35:36.126 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2231.g2edfee68 2024-03-21T08:35:36.127 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-21T08:35:36.133 INFO:teuthology.task.internal:no buildpackages task found 2024-03-21T08:35:36.133 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-21T08:35:36.138 INFO:teuthology.task.internal:Saving configuration 2024-03-21T08:35:36.158 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-21T08:35:36.164 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T08:35:36.190 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:30:32.639937', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfQ5mJ3bWnXAMBSktocr6hXpZC+vE9PwUKFZLqI7jI+ew4wzW7MsSXLDX2ELpAhsDehpUg+emWycV7YwX/Zhko='} 2024-03-21T08:35:36.215 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:30:32.635996', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVVoaOBkE3CItZzR+zdxw5fQRlolyR8GPnQhe1YvWnJGeWD09s1hxpKJTGnOuvZmQDkItj4YHrSLawE8hZrMVE='} 2024-03-21T08:35:36.238 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:30:32.634397', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4zlef7E7np1ld4jsMgNcjLRPZ+Q3Tck4iGeYrvwXb0VawcGgLDDTeIUkYSQnqH27s6fqHa2Ijd5Utipw+2dZA='} 2024-03-21T08:35:36.238 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-21T08:35:36.244 INFO:teuthology.task.internal:roles: ubuntu@smithi087.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-03-21T08:35:36.245 INFO:teuthology.task.internal:roles: ubuntu@smithi100.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-03-21T08:35:36.245 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-03-21T08:35:36.245 INFO:teuthology.run_tasks:Running task console_log... 2024-03-21T08:35:36.354 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f96ca759160>, signals=[15]) 2024-03-21T08:35:36.355 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-21T08:35:36.362 INFO:teuthology.task.internal:Opening connections... 2024-03-21T08:35:36.362 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi087.front.sepia.ceph.com 2024-03-21T08:35:36.364 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:35:36.456 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi100.front.sepia.ceph.com 2024-03-21T08:35:36.457 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:35:36.543 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-03-21T08:35:36.544 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:35:36.633 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-21T08:35:36.639 DEBUG:teuthology.orchestra.run.smithi087:> uname -m 2024-03-21T08:35:36.657 INFO:teuthology.orchestra.run.smithi087.stdout:x86_64 2024-03-21T08:35:36.657 DEBUG:teuthology.orchestra.run.smithi087:> cat /etc/os-release 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:NAME="CentOS Stream" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION="9" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:ID="centos" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:VERSION_ID="9" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:ANSI_COLOR="0;31" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:35:36.714 INFO:teuthology.orchestra.run.smithi087.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:35:36.715 INFO:teuthology.orchestra.run.smithi087.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:35:36.715 INFO:teuthology.orchestra.run.smithi087.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:35:36.715 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:35:36.715 INFO:teuthology.orchestra.run.smithi087.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:35:36.715 INFO:teuthology.lock.ops:Updating smithi087.front.sepia.ceph.com on lock server 2024-03-21T08:35:36.739 DEBUG:teuthology.orchestra.run.smithi100:> uname -m 2024-03-21T08:35:36.757 INFO:teuthology.orchestra.run.smithi100.stdout:x86_64 2024-03-21T08:35:36.757 DEBUG:teuthology.orchestra.run.smithi100:> cat /etc/os-release 2024-03-21T08:35:36.814 INFO:teuthology.orchestra.run.smithi100.stdout:NAME="CentOS Stream" 2024-03-21T08:35:36.814 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION="9" 2024-03-21T08:35:36.814 INFO:teuthology.orchestra.run.smithi100.stdout:ID="centos" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:VERSION_ID="9" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:ANSI_COLOR="0;31" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:35:36.815 INFO:teuthology.orchestra.run.smithi100.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:35:36.816 INFO:teuthology.lock.ops:Updating smithi100.front.sepia.ceph.com on lock server 2024-03-21T08:35:36.839 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-03-21T08:35:36.856 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-03-21T08:35:36.857 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-03-21T08:35:36.914 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="CentOS Stream" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="9" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:ID="centos" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE="rhel fedora" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="9" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:PLATFORM_ID="platform:el9" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="CentOS Stream 9" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:ANSI_COLOR="0;31" 2024-03-21T08:35:36.915 INFO:teuthology.orchestra.run.smithi105.stdout:LOGO="fedora-logo-icon" 2024-03-21T08:35:36.916 INFO:teuthology.orchestra.run.smithi105.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-03-21T08:35:36.916 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://centos.org/" 2024-03-21T08:35:36.916 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-03-21T08:35:36.916 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-03-21T08:35:36.916 INFO:teuthology.orchestra.run.smithi105.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-03-21T08:35:36.916 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-03-21T08:35:36.939 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-21T08:35:36.950 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-21T08:35:36.955 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-21T08:35:36.956 DEBUG:teuthology.orchestra.run.smithi087:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:35:36.958 DEBUG:teuthology.orchestra.run.smithi100:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:35:36.960 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-03-21T08:35:36.977 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-21T08:35:36.983 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-21T08:35:36.983 DEBUG:teuthology.orchestra.run.smithi087:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:35:37.014 DEBUG:teuthology.orchestra.run.smithi100:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:35:37.016 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-03-21T08:35:37.041 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-21T08:35:37.120 INFO:teuthology.run_tasks:Running task kernel... 2024-03-21T08:35:37.136 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '5ec3e6e88b2e2783e09498588bb1f433ae9c8986'} 2024-03-21T08:35:37.136 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-03-21T08:35:37.136 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-03-21T08:35:37.136 DEBUG:teuthology.orchestra.run.smithi087:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:35:37.153 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:35:37.154 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-03-21T08:35:37.211 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-361.el9.x86_64 2024-03-21T08:35:37.211 INFO:teuthology.task.kernel:Running kernel on smithi087: 5.14.0-361.el9.x86_64 2024-03-21T08:35:37.211 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum install -y kernel 2024-03-21T08:35:39.322 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - AppStream 15 MB/s | 6.7 MB 00:00 2024-03-21T08:35:40.947 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - BaseOS 3.8 MB/s | 2.0 MB 00:00 2024-03-21T08:35:45.788 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.0 MB 00:04 2024-03-21T08:35:49.463 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 7.2 MB/s | 19 MB 00:02 2024-03-21T08:35:53.322 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.1 MB 00:00 2024-03-21T08:35:54.965 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-03-21T08:35:57.127 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-03-21T08:36:02.016 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 15 kB/s | 1.7 kB 00:00 2024-03-21T08:36:02.183 INFO:teuthology.orchestra.run.smithi087.stdout:Last metadata expiration check: 0:00:01 ago on Thu 21 Mar 2024 08:36:01 AM UTC. 2024-03-21T08:36:04.031 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-03-21T08:36:04.031 INFO:teuthology.orchestra.run.smithi087.stdout:Package kernel-5.14.0-361.el9.x86_64 is already installed. 2024-03-21T08:36:04.140 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-03-21T08:36:04.152 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-03-21T08:36:04.152 INFO:teuthology.orchestra.run.smithi087.stdout: Package Architecture Version Repository Size 2024-03-21T08:36:04.152 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-03-21T08:36:04.152 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout: kernel x86_64 5.14.0-430.el9 baseos 44 k 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core x86_64 5.14.0-430.el9 baseos 15 M 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules x86_64 5.14.0-430.el9 baseos 33 M 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core x86_64 5.14.0-430.el9 baseos 27 M 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout:================================================================================ 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout:Install 4 Packages 2024-03-21T08:36:04.153 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:04.154 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 76 M 2024-03-21T08:36:04.154 INFO:teuthology.orchestra.run.smithi087.stdout:Installed size: 123 M 2024-03-21T08:36:04.154 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-03-21T08:36:04.517 INFO:teuthology.orchestra.run.smithi087.stdout:(1/4): kernel-5.14.0-430.el9.x86_64.rpm 182 kB/s | 44 kB 00:00 2024-03-21T08:36:05.402 INFO:teuthology.orchestra.run.smithi087.stdout:(2/4): kernel-modules-5.14.0-430.el9.x86_64.rpm 30 MB/s | 33 MB 00:01 2024-03-21T08:36:05.818 INFO:teuthology.orchestra.run.smithi087.stdout:(3/4): kernel-core-5.14.0-430.el9.x86_64.rpm 9.7 MB/s | 15 MB 00:01 2024-03-21T08:36:06.560 INFO:teuthology.orchestra.run.smithi087.stdout:(4/4): kernel-modules-core-5.14.0-430.el9.x86_6 13 MB/s | 27 MB 00:02 2024-03-21T08:36:06.562 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:36:06.563 INFO:teuthology.orchestra.run.smithi087.stdout:Total 31 MB/s | 76 MB 00:02 2024-03-21T08:36:07.224 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-03-21T08:36:07.583 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-03-21T08:36:07.583 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-03-21T08:36:08.573 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-03-21T08:36:08.574 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-03-21T08:36:11.056 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-03-21T08:36:11.655 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-core-5.14.0-430.el9.x86_64 1/4 2024-03-21T08:36:11.743 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T08:36:14.489 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T08:36:14.614 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T08:36:19.357 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T08:36:19.397 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T08:36:24.108 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T08:36:53.606 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T08:36:53.607 INFO:teuthology.orchestra.run.smithi087.stdout:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2024-03-21T08:36:53.607 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:53.651 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-modules-5.14.0-430.el9.x86_64 4/4 2024-03-21T08:36:53.998 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: kernel-5.14.0-430.el9.x86_64 4/4 2024-03-21T08:36:53.998 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-5.14.0-430.el9.x86_64 1/4 2024-03-21T08:36:53.998 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-core-5.14.0-430.el9.x86_64 2/4 2024-03-21T08:36:53.998 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-5.14.0-430.el9.x86_64 3/4 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : kernel-modules-core-5.14.0-430.el9.x86_64 4/4 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-5.14.0-430.el9.x86_64 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-core-5.14.0-430.el9.x86_64 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-5.14.0-430.el9.x86_64 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: kernel-modules-core-5.14.0-430.el9.x86_64 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:54.391 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-03-21T08:36:54.699 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T08:36:54.796 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T08:36:54.797 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-361.el9.x86_64 Expected: 5.14.0-430.el9.x86_64 2024-03-21T08:36:54.797 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-21T08:36:54.797 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-21T08:36:54.797 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-03-21T08:36:54.797 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi087.front.sepia.ceph.com, path=None, version=distro) 2024-03-21T08:36:54.797 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q kernel | sort -rV | head -n 1 2024-03-21T08:36:54.876 INFO:teuthology.orchestra.run.smithi087.stdout:kernel-5.14.0-430.el9.x86_64 2024-03-21T08:36:54.876 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-430.el9.x86_64 2024-03-21T08:36:54.876 DEBUG:teuthology.orchestra.run.smithi087:> sudo rpm -qi grub2-tools 2024-03-21T08:36:54.910 INFO:teuthology.orchestra.run.smithi087.stdout:Name : grub2-tools 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Epoch : 1 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Version : 2.06 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Release : 68.el9 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Architecture: x86_64 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Group : Unspecified 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Size : 8264254 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:License : GPLv3+ 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Packager : builder@centos.org 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:Vendor : CentOS 2024-03-21T08:36:54.911 INFO:teuthology.orchestra.run.smithi087.stdout:URL : http://www.gnu.org/software/grub/ 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:Summary : Support tools for GRUB. 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:Description : 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:customizable bootloader with modular architecture. It supports a rich 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:variety of kernel formats, file systems, computer architectures and 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:hardware devices. 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:36:54.912 INFO:teuthology.orchestra.run.smithi087.stdout:This subpackage provides tools for support of all platforms. 2024-03-21T08:36:54.913 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-03-21T08:36:54.913 INFO:teuthology.task.kernel:Updating grub on smithi087 to boot 5.14.0-430.el9.x86_64 2024-03-21T08:36:54.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-03-21T08:36:55.520 INFO:teuthology.orchestra.run.smithi087.stderr:Generating grub configuration file ... 2024-03-21T08:36:56.842 INFO:teuthology.orchestra.run.smithi087.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-03-21T08:36:56.876 INFO:teuthology.orchestra.run.smithi087.stderr:done 2024-03-21T08:36:56.878 DEBUG:teuthology.orchestra.run.smithi087:> mktemp 2024-03-21T08:36:56.893 INFO:teuthology.orchestra.run.smithi087.stdout:/tmp/tmp.aOyHEgJptL 2024-03-21T08:36:56.894 DEBUG:teuthology.orchestra.run.smithi087:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.aOyHEgJptL 2024-03-21T08:36:56.962 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0666 /tmp/tmp.aOyHEgJptL 2024-03-21T08:36:57.107 DEBUG:teuthology.orchestra.remote:smithi087:/tmp/tmp.aOyHEgJptL is 6KB 2024-03-21T08:36:57.156 DEBUG:teuthology.orchestra.run.smithi087:> rm -fr /tmp/tmp.aOyHEgJptL 2024-03-21T08:36:57.170 DEBUG:teuthology.orchestra.run.smithi087:> sudo /bin/ls /boot/loader/entries || true 2024-03-21T08:36:57.235 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2024-03-21T08:36:57.235 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2024-03-21T08:36:57.235 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2024-03-21T08:36:57.236 INFO:teuthology.orchestra.run.smithi087.stdout:15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64.conf 2024-03-21T08:36:57.237 DEBUG:teuthology.orchestra.run.smithi087:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-5.14.0-430.el9.x86_64 2024-03-21T08:36:57.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo shutdown -r now 2024-03-21T08:37:27.390 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-03-21T08:37:27.391 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-21T08:37:27.392 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:37:45.886 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.87 2024-03-21T08:37:54.887 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-21T08:37:54.888 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:38:54.889 DEBUG:teuthology.orchestra.remote:timed out 2024-03-21T08:39:06.890 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-21T08:39:06.891 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:39:07.168 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-03-21T08:39:07.530 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-21T08:39:07.531 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-430.el9.x86_64"... 2024-03-21T08:39:07.531 DEBUG:teuthology.orchestra.run.smithi087:> uname -r 2024-03-21T08:39:07.546 INFO:teuthology.orchestra.run.smithi087.stdout:5.14.0-430.el9.x86_64 2024-03-21T08:39:07.547 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-430.el9.x86_64 vs 5.14.0-430.el9.x86_64 2024-03-21T08:39:07.547 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-21T08:39:07.547 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-03-21T08:39:08.547 DEBUG:teuthology.parallel:result is None 2024-03-21T08:39:08.548 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-21T08:39:08.555 INFO:teuthology.task.internal:Creating test directory... 2024-03-21T08:39:08.555 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:39:08.557 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:39:08.559 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-21T08:39:08.577 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-21T08:39:08.603 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-21T08:39:08.612 INFO:teuthology.task.internal:Creating archive directory... 2024-03-21T08:39:08.612 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:39:08.614 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:39:08.618 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-21T08:39:08.863 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-21T08:39:08.870 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-21T08:39:08.870 DEBUG:teuthology.orchestra.run.smithi100:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:39:08.887 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:39:08.887 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-21T08:39:08.903 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:39:08.904 DEBUG:teuthology.orchestra.run.smithi087:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:39:08.906 DEBUG:teuthology.orchestra.run.smithi100:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:39:08.930 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-21T08:39:08.991 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:39:09.001 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:39:09.002 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:39:09.012 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:39:09.027 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:39:09.046 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-21T08:39:09.048 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-21T08:39:09.056 INFO:teuthology.task.internal:Configuring sudo... 2024-03-21T08:39:09.056 DEBUG:teuthology.orchestra.run.smithi087:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:39:09.058 DEBUG:teuthology.orchestra.run.smithi100:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:39:09.061 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-21T08:39:09.093 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-21T08:39:09.102 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-21T08:39:09.102 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:39:09.129 DEBUG:teuthology.orchestra.run.smithi100:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:39:09.131 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-21T08:39:09.149 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:39:09.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:39:09.290 DEBUG:teuthology.orchestra.run.smithi087:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:39:09.331 DEBUG:teuthology.orchestra.run.smithi087:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:39:09.375 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:39:09.375 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:39:09.443 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:39:09.488 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:39:09.542 DEBUG:teuthology.orchestra.run.smithi100:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:39:09.626 DEBUG:teuthology.orchestra.run.smithi100:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:39:09.671 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:39:09.671 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:39:09.740 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:39:09.786 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-21T08:39:09.839 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:39:09.923 DEBUG:teuthology.orchestra.run.smithi105:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-21T08:39:09.969 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:39:09.970 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-21T08:39:10.039 DEBUG:teuthology.orchestra.run.smithi087:> sudo service rsyslog restart 2024-03-21T08:39:10.042 DEBUG:teuthology.orchestra.run.smithi100:> sudo service rsyslog restart 2024-03-21T08:39:10.044 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-03-21T08:39:10.079 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:39:10.097 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:39:10.121 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T08:39:10.548 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-21T08:39:10.558 INFO:teuthology.task.internal:Starting timer... 2024-03-21T08:39:10.559 INFO:teuthology.run_tasks:Running task pcp... 2024-03-21T08:39:10.569 INFO:teuthology.run_tasks:Running task selinux... 2024-03-21T08:39:10.579 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-03-21T08:39:10.580 DEBUG:teuthology.orchestra.run.smithi087:> sudo service auditd rotate 2024-03-21T08:39:10.637 INFO:teuthology.orchestra.run.smithi087.stdout:Rotating logs: 2024-03-21T08:39:10.639 DEBUG:teuthology.orchestra.run.smithi100:> sudo service auditd rotate 2024-03-21T08:39:10.702 INFO:teuthology.orchestra.run.smithi100.stdout:Rotating logs: 2024-03-21T08:39:10.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo service auditd rotate 2024-03-21T08:39:10.760 INFO:teuthology.orchestra.run.smithi105.stdout:Rotating logs: 2024-03-21T08:39:10.761 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-21T08:39:10.762 DEBUG:teuthology.orchestra.run.smithi087:> /usr/sbin/getenforce 2024-03-21T08:39:10.781 INFO:teuthology.orchestra.run.smithi087.stdout:Permissive 2024-03-21T08:39:10.781 DEBUG:teuthology.orchestra.run.smithi100:> /usr/sbin/getenforce 2024-03-21T08:39:10.806 INFO:teuthology.orchestra.run.smithi100.stdout:Permissive 2024-03-21T08:39:10.806 DEBUG:teuthology.orchestra.run.smithi105:> /usr/sbin/getenforce 2024-03-21T08:39:10.826 INFO:teuthology.orchestra.run.smithi105.stdout:Permissive 2024-03-21T08:39:10.826 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi087.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi100.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi105.front.sepia.ceph.com': 'permissive'} 2024-03-21T08:39:10.826 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:39:10.856 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:39:10.856 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:39:10.884 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:39:10.885 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T08:39:10.912 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:39:10.913 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-21T08:39:10.913 DEBUG:teuthology.orchestra.run.smithi087:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:39:10.958 DEBUG:teuthology.orchestra.run.smithi100:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:39:10.991 DEBUG:teuthology.orchestra.run.smithi105:> sudo /usr/sbin/setenforce permissive 2024-03-21T08:39:11.027 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-21T08:39:11.037 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-21T08:39:11.042 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-21T08:39:11.180 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-21T08:39:11.196 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-21T08:39:11.198 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi087.front.sepia.ceph.com,smithi100.front.sepia.ceph.com,smithi105.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-21T08:44:08.911 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi087.front.sepia.ceph.com'), Remote(name='ubuntu@smithi100.front.sepia.ceph.com'), Remote(name='ubuntu@smithi105.front.sepia.ceph.com')] 2024-03-21T08:44:08.913 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-21T08:44:08.914 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi087.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:09.006 DEBUG:teuthology.orchestra.run.smithi087:> true 2024-03-21T08:44:09.041 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi087.front.sepia.ceph.com' 2024-03-21T08:44:09.041 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T08:44:09.042 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi100.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:09.127 DEBUG:teuthology.orchestra.run.smithi100:> true 2024-03-21T08:44:09.192 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi100.front.sepia.ceph.com' 2024-03-21T08:44:09.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-21T08:44:09.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-21T08:44:09.282 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-03-21T08:44:09.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-03-21T08:44:09.351 INFO:teuthology.run_tasks:Running task clock... 2024-03-21T08:44:09.361 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-21T08:44:09.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:44:09.362 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:44:09.364 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:44:09.364 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:44:09.366 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-21T08:44:09.366 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T08:44:09.394 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:44:09.394 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:44:09.411 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:44:09.411 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:44:09.421 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-03-21T08:44:09.438 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-03-21T08:44:09.440 INFO:teuthology.orchestra.run.smithi100.stderr:sudo: ntpd: command not found 2024-03-21T08:44:09.441 INFO:teuthology.orchestra.run.smithi087.stderr:sudo: ntpd: command not found 2024-03-21T08:44:09.454 INFO:teuthology.orchestra.run.smithi100.stdout:506 Cannot talk to daemon 2024-03-21T08:44:09.454 INFO:teuthology.orchestra.run.smithi087.stdout:506 Cannot talk to daemon 2024-03-21T08:44:09.466 INFO:teuthology.orchestra.run.smithi105.stderr:sudo: ntpd: command not found 2024-03-21T08:44:09.471 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:44:09.471 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:44:09.480 INFO:teuthology.orchestra.run.smithi105.stdout:506 Cannot talk to daemon 2024-03-21T08:44:09.487 INFO:teuthology.orchestra.run.smithi100.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:44:09.487 INFO:teuthology.orchestra.run.smithi087.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:44:09.496 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-03-21T08:44:09.512 INFO:teuthology.orchestra.run.smithi105.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-03-21T08:44:09.530 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:44:09.534 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:44:09.535 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-21T08:44:09.535 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.535 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.535 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.535 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.543 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:44:09.546 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:44:09.547 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-03-21T08:44:09.547 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.547 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.547 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.547 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.555 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-21T08:44:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T08:44:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-21T08:44:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.559 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T08:44:09.559 INFO:teuthology.run_tasks:Running task install... 2024-03-21T08:44:09.567 DEBUG:teuthology.task.install:project ceph 2024-03-21T08:44:09.567 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'}} 2024-03-21T08:44:09.567 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11'} 2024-03-21T08:44:09.568 INFO:teuthology.task.install:Using flavor: default 2024-03-21T08:44:09.575 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-21T08:44:09.575 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-03-21T08:44:09.575 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:44:09.575 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:44:09.577 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:44:09.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:44:09.578 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'tag': None, 'wait_for_package': False} 2024-03-21T08:44:09.578 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:44:09.719 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:44:09.719 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:44:09.739 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:44:09.739 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:44:09.752 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/ 2024-03-21T08:44:09.752 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2231.g2edfee68 2024-03-21T08:44:09.827 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:44:09.827 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:44:09.827 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:44:09.846 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:44:09.846 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:44:09.846 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:44:09.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:44:09.855 DEBUG:teuthology.orchestra.run.smithi087:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:44:09.874 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:44:09.874 DEBUG:teuthology.orchestra.run.smithi105:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:44:09.881 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-03-21T08:44:09.881 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:44:09.881 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-03-21T08:44:09.908 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-03-21T08:44:09.909 DEBUG:teuthology.orchestra.run.smithi100:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2edfee68ad7243bed422a049b6f5f58619300f11/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-03-21T08:44:09.922 DEBUG:teuthology.orchestra.run.smithi087:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:44:09.942 DEBUG:teuthology.orchestra.run.smithi105:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:44:09.976 DEBUG:teuthology.orchestra.run.smithi100:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-03-21T08:44:10.001 DEBUG:teuthology.orchestra.run.smithi087:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:44:10.021 DEBUG:teuthology.orchestra.run.smithi105:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:44:10.056 DEBUG:teuthology.orchestra.run.smithi100:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-03-21T08:44:10.067 INFO:teuthology.orchestra.run.smithi087.stdout:check_obsoletes = 1 2024-03-21T08:44:10.069 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum clean all 2024-03-21T08:44:10.088 INFO:teuthology.orchestra.run.smithi105.stdout:check_obsoletes = 1 2024-03-21T08:44:10.089 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum clean all 2024-03-21T08:44:10.121 INFO:teuthology.orchestra.run.smithi100.stdout:check_obsoletes = 1 2024-03-21T08:44:10.123 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum clean all 2024-03-21T08:44:10.307 INFO:teuthology.orchestra.run.smithi087.stdout:74 files removed 2024-03-21T08:44:10.322 INFO:teuthology.orchestra.run.smithi105.stdout:74 files removed 2024-03-21T08:44:10.334 DEBUG:teuthology.orchestra.run.smithi087:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:44:10.349 DEBUG:teuthology.orchestra.run.smithi105:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:44:10.358 INFO:teuthology.orchestra.run.smithi100.stdout:74 files removed 2024-03-21T08:44:10.385 DEBUG:teuthology.orchestra.run.smithi100:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-03-21T08:44:10.964 INFO:teuthology.orchestra.run.smithi087.stdout:ceph packages for x86_64 249 kB/s | 83 kB 00:00 2024-03-21T08:44:10.971 INFO:teuthology.orchestra.run.smithi105.stdout:ceph packages for x86_64 248 kB/s | 83 kB 00:00 2024-03-21T08:44:11.009 INFO:teuthology.orchestra.run.smithi100.stdout:ceph packages for x86_64 249 kB/s | 83 kB 00:00 2024-03-21T08:44:11.297 INFO:teuthology.orchestra.run.smithi105.stdout:ceph noarch packages 35 kB/s | 10 kB 00:00 2024-03-21T08:44:11.298 INFO:teuthology.orchestra.run.smithi087.stdout:ceph noarch packages 35 kB/s | 10 kB 00:00 2024-03-21T08:44:11.343 INFO:teuthology.orchestra.run.smithi100.stdout:ceph noarch packages 34 kB/s | 10 kB 00:00 2024-03-21T08:44:11.598 INFO:teuthology.orchestra.run.smithi105.stdout:ceph source packages 7.0 kB/s | 1.9 kB 00:00 2024-03-21T08:44:11.641 INFO:teuthology.orchestra.run.smithi087.stdout:ceph source packages 6.2 kB/s | 1.9 kB 00:00 2024-03-21T08:44:11.651 INFO:teuthology.orchestra.run.smithi100.stdout:ceph source packages 6.8 kB/s | 1.9 kB 00:00 2024-03-21T08:44:12.009 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - AppStream 17 MB/s | 6.7 MB 00:00 2024-03-21T08:44:12.171 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - AppStream 14 MB/s | 6.7 MB 00:00 2024-03-21T08:44:12.219 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - AppStream 12 MB/s | 6.7 MB 00:00 2024-03-21T08:44:13.376 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS-9 - BaseOS 7.7 MB/s | 2.0 MB 00:00 2024-03-21T08:44:13.596 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS-9 - BaseOS 6.5 MB/s | 2.0 MB 00:00 2024-03-21T08:44:13.702 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS-9 - BaseOS 5.3 MB/s | 2.0 MB 00:00 2024-03-21T08:44:14.493 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.0 MB 00:00 2024-03-21T08:44:14.545 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.0 MB 00:00 2024-03-21T08:44:16.333 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - BaseOS 3.1 MB/s | 8.0 MB 00:02 2024-03-21T08:44:16.386 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-03-21T08:44:17.155 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-03-21T08:44:18.376 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-03-21T08:44:21.741 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.1 MB 00:00 2024-03-21T08:44:23.376 INFO:teuthology.orchestra.run.smithi100.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-03-21T08:44:24.891 INFO:teuthology.orchestra.run.smithi100.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-03-21T08:44:27.636 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - CRB 804 kB/s | 6.1 MB 00:07 2024-03-21T08:44:28.961 INFO:teuthology.orchestra.run.smithi087.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 16 kB 00:00 2024-03-21T08:44:29.716 INFO:teuthology.orchestra.run.smithi100.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-03-21T08:44:30.607 INFO:teuthology.orchestra.run.smithi087.stdout:Extra Packages for Enterprise Linux 14 MB/s | 21 MB 00:01 2024-03-21T08:44:32.044 INFO:teuthology.orchestra.run.smithi100.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:44:32.044 INFO:teuthology.orchestra.run.smithi100.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:44:32.057 INFO:teuthology.orchestra.run.smithi100.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:44:32.058 INFO:teuthology.orchestra.run.smithi100.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:44:32.059 INFO:teuthology.orchestra.run.smithi100.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:44:32.059 INFO:teuthology.orchestra.run.smithi100.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:44:32.061 INFO:teuthology.orchestra.run.smithi100.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:44:32.064 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:44:32.065 INFO:teuthology.orchestra.run.smithi100.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:44:32.145 INFO:teuthology.orchestra.run.smithi100.stdout:Dependencies resolved. 2024-03-21T08:44:32.156 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T08:44:32.156 INFO:teuthology.orchestra.run.smithi100.stdout: Package Arch Version Repository Size 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout:Installing: 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:44:32.157 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:44:32.158 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:44:32.158 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:44:32.158 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:44:32.158 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:44:32.161 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:44:32.161 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:44:32.161 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:44:32.162 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrading: 2024-03-21T08:44:32.163 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:44:32.164 INFO:teuthology.orchestra.run.smithi100.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:44:32.165 INFO:teuthology.orchestra.run.smithi100.stdout:Installing dependencies: 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:44:32.166 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:44:32.167 INFO:teuthology.orchestra.run.smithi100.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:44:32.168 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:44:32.169 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:44:32.170 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:44:32.171 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:44:32.172 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:44:32.173 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:44:32.174 INFO:teuthology.orchestra.run.smithi100.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout:Installing weak dependencies: 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction Summary 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout:============================================================================================ 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout:Install 150 Packages 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout:Upgrade 24 Packages 2024-03-21T08:44:32.175 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:44:32.176 INFO:teuthology.orchestra.run.smithi100.stdout:Total download size: 224 M 2024-03-21T08:44:32.176 INFO:teuthology.orchestra.run.smithi100.stdout:Downloading Packages: 2024-03-21T08:44:32.558 INFO:teuthology.orchestra.run.smithi100.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 35 kB/s | 6.6 kB 00:00 2024-03-21T08:44:32.742 INFO:teuthology.orchestra.run.smithi100.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 4.7 MB/s | 890 kB 00:00 2024-03-21T08:44:32.801 INFO:teuthology.orchestra.run.smithi100.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 2.4 MB/s | 145 kB 00:00 2024-03-21T08:44:33.151 INFO:teuthology.orchestra.run.smithi100.stdout:(4/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 6.6 MB/s | 2.3 MB 00:00 2024-03-21T08:44:33.226 INFO:teuthology.orchestra.run.smithi100.stdout:(5/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 6.2 MB/s | 5.4 MB 00:00 2024-03-21T08:44:33.426 INFO:teuthology.orchestra.run.smithi100.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 5.4 MB/s | 1.5 MB 00:00 2024-03-21T08:44:33.743 INFO:teuthology.orchestra.run.smithi100.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 9.1 MB/s | 4.7 MB 00:00 2024-03-21T08:44:34.594 INFO:teuthology.orchestra.run.smithi100.stdout:(8/174): ceph-common-19.0.0-2231.g2edfee68.el9. 9.2 MB/s | 21 MB 00:02 2024-03-21T08:44:34.652 INFO:teuthology.orchestra.run.smithi100.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 433 kB/s | 25 kB 00:00 2024-03-21T08:44:34.853 INFO:teuthology.orchestra.run.smithi100.stdout:(10/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 8.9 MB/s | 9.9 MB 00:01 2024-03-21T08:44:34.928 INFO:teuthology.orchestra.run.smithi100.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 2.9 MB/s | 220 kB 00:00 2024-03-21T08:44:34.983 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - CRB 487 kB/s | 6.1 MB 00:12 2024-03-21T08:44:34.986 INFO:teuthology.orchestra.run.smithi100.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 465 kB/s | 27 kB 00:00 2024-03-21T08:44:35.045 INFO:teuthology.orchestra.run.smithi100.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 537 kB/s | 31 kB 00:00 2024-03-21T08:44:35.195 INFO:teuthology.orchestra.run.smithi100.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 4.9 MB/s | 747 kB 00:00 2024-03-21T08:44:35.279 INFO:teuthology.orchestra.run.smithi100.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.9 MB/s | 163 kB 00:00 2024-03-21T08:44:35.463 INFO:teuthology.orchestra.run.smithi100.stdout:(16/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 8.3 MB/s | 17 MB 00:02 2024-03-21T08:44:35.463 INFO:teuthology.orchestra.run.smithi087.stdout:lab-extras 42 kB/s | 1.7 kB 00:00 2024-03-21T08:44:35.488 INFO:teuthology.orchestra.run.smithi100.stdout:(17/174): librados-devel-19.0.0-2231.g2edfee68. 607 kB/s | 127 kB 00:00 2024-03-21T08:44:35.521 INFO:teuthology.orchestra.run.smithi100.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 8.4 MB/s | 501 kB 00:00 2024-03-21T08:44:35.592 INFO:teuthology.orchestra.run.smithi100.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 645 kB/s | 45 kB 00:00 2024-03-21T08:44:35.675 INFO:teuthology.orchestra.run.smithi100.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.5 MB/s | 131 kB 00:00 2024-03-21T08:44:35.759 INFO:teuthology.orchestra.run.smithi100.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 1.9 MB/s | 161 kB 00:00 2024-03-21T08:44:35.860 INFO:teuthology.orchestra.run.smithi100.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 3.1 MB/s | 320 kB 00:00 2024-03-21T08:44:35.952 INFO:teuthology.orchestra.run.smithi100.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 3.2 MB/s | 297 kB 00:00 2024-03-21T08:44:36.027 INFO:teuthology.orchestra.run.smithi100.stdout:(24/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 9.6 MB/s | 5.1 MB 00:00 2024-03-21T08:44:36.054 INFO:teuthology.orchestra.run.smithi100.stdout:(25/174): python3-rgw-19.0.0-2231.g2edfee68.el9 993 kB/s | 99 kB 00:00 2024-03-21T08:44:36.086 INFO:teuthology.orchestra.run.smithi100.stdout:(26/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.4 MB/s | 85 kB 00:00 2024-03-21T08:44:36.170 INFO:teuthology.orchestra.run.smithi100.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.0 MB/s | 171 kB 00:00 2024-03-21T08:44:36.237 INFO:teuthology.orchestra.run.smithi100.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 510 kB/s | 34 kB 00:00 2024-03-21T08:44:36.312 INFO:teuthology.orchestra.run.smithi100.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.8 MB/s | 140 kB 00:00 2024-03-21T08:44:36.412 INFO:teuthology.orchestra.run.smithi100.stdout:(30/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 8.6 MB/s | 3.1 MB 00:00 2024-03-21T08:44:36.525 INFO:teuthology.orchestra.run.smithi105.stdout:CentOS Stream 9 - Extras packages 38 kB/s | 16 kB 00:00 2024-03-21T08:44:36.546 INFO:teuthology.orchestra.run.smithi100.stdout:(31/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 7.7 MB/s | 1.8 MB 00:00 2024-03-21T08:44:36.646 INFO:teuthology.orchestra.run.smithi100.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.4 MB/s | 248 kB 00:00 2024-03-21T08:44:36.730 INFO:teuthology.orchestra.run.smithi100.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 594 kB/s | 49 kB 00:00 2024-03-21T08:44:36.805 INFO:teuthology.orchestra.run.smithi100.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 219 kB/s | 16 kB 00:00 2024-03-21T08:44:36.905 INFO:teuthology.orchestra.run.smithi100.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.6 MB/s | 264 kB 00:00 2024-03-21T08:44:37.089 INFO:teuthology.orchestra.run.smithi100.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 4.0 MB/s | 750 kB 00:00 2024-03-21T08:44:37.172 INFO:teuthology.orchestra.run.smithi100.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 219 kB/s | 18 kB 00:00 2024-03-21T08:44:37.323 INFO:teuthology.orchestra.run.smithi100.stdout:(38/174): ceph-mgr-diskprediction-local-19.0.0- 8.1 MB/s | 7.4 MB 00:00 2024-03-21T08:44:37.406 INFO:teuthology.orchestra.run.smithi100.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 2.8 MB/s | 662 kB 00:00 2024-03-21T08:44:37.440 INFO:teuthology.orchestra.run.smithi100.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 7.7 MB/s | 920 kB 00:00 2024-03-21T08:44:37.473 INFO:teuthology.orchestra.run.smithi100.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-21T08:44:37.499 INFO:teuthology.orchestra.run.smithi100.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-21T08:44:37.524 INFO:teuthology.orchestra.run.smithi100.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 638 kB/s | 32 kB 00:00 2024-03-21T08:44:37.619 INFO:teuthology.orchestra.run.smithi105.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-03-21T08:44:37.876 INFO:teuthology.orchestra.run.smithi087.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:44:37.877 INFO:teuthology.orchestra.run.smithi087.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:44:37.894 INFO:teuthology.orchestra.run.smithi087.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:44:37.894 INFO:teuthology.orchestra.run.smithi087.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:44:37.894 INFO:teuthology.orchestra.run.smithi087.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:44:37.895 INFO:teuthology.orchestra.run.smithi087.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:44:37.897 INFO:teuthology.orchestra.run.smithi087.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:44:37.901 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:44:37.902 INFO:teuthology.orchestra.run.smithi087.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:44:37.966 INFO:teuthology.orchestra.run.smithi100.stdout:(44/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 14 MB/s | 48 MB 00:03 2024-03-21T08:44:37.982 INFO:teuthology.orchestra.run.smithi087.stdout:Dependencies resolved. 2024-03-21T08:44:38.004 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================ 2024-03-21T08:44:38.004 INFO:teuthology.orchestra.run.smithi087.stdout: Package Arch Version Repository Size 2024-03-21T08:44:38.004 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================ 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout:Installing: 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:44:38.005 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:44:38.006 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:44:38.007 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrading: 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:44:38.008 INFO:teuthology.orchestra.run.smithi087.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout:Installing dependencies: 2024-03-21T08:44:38.009 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:44:38.010 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:44:38.011 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:44:38.012 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:44:38.013 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:44:38.014 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:44:38.015 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:44:38.016 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:44:38.017 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout:Installing weak dependencies: 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction Summary 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout:============================================================================================ 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout:Install 150 Packages 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout:Upgrade 24 Packages 2024-03-21T08:44:38.018 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:44:38.019 INFO:teuthology.orchestra.run.smithi100.stdout:(45/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-03-21T08:44:38.019 INFO:teuthology.orchestra.run.smithi087.stdout:Total download size: 224 M 2024-03-21T08:44:38.019 INFO:teuthology.orchestra.run.smithi087.stdout:Downloading Packages: 2024-03-21T08:44:38.041 INFO:teuthology.orchestra.run.smithi100.stdout:(46/174): flexiblas-openblas-openmp-3.0.4-8.el9 33 kB/s | 17 kB 00:00 2024-03-21T08:44:38.075 INFO:teuthology.orchestra.run.smithi100.stdout:(47/174): libaio-devel-0.3.111-13.el9.x86_64.rp 94 kB/s | 10 kB 00:00 2024-03-21T08:44:38.100 INFO:teuthology.orchestra.run.smithi100.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 760 kB/s | 44 kB 00:00 2024-03-21T08:44:38.126 INFO:teuthology.orchestra.run.smithi100.stdout:(49/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2024-03-21T08:44:38.160 INFO:teuthology.orchestra.run.smithi100.stdout:(50/174): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 661 kB 00:00 2024-03-21T08:44:38.185 INFO:teuthology.orchestra.run.smithi100.stdout:(51/174): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 578 kB 00:00 2024-03-21T08:44:38.210 INFO:teuthology.orchestra.run.smithi100.stdout:(52/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 946 kB/s | 46 kB 00:00 2024-03-21T08:44:38.235 INFO:teuthology.orchestra.run.smithi100.stdout:(53/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.7 MB/s | 282 kB 00:00 2024-03-21T08:44:38.261 INFO:teuthology.orchestra.run.smithi100.stdout:(54/174): lua-5.4.4-4.el9.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2024-03-21T08:44:38.286 INFO:teuthology.orchestra.run.smithi100.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 688 kB/s | 34 kB 00:00 2024-03-21T08:44:38.311 INFO:teuthology.orchestra.run.smithi100.stdout:(56/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.8 MB/s | 217 kB 00:00 2024-03-21T08:44:38.453 INFO:teuthology.orchestra.run.smithi100.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 25 MB/s | 4.7 MB 00:00 2024-03-21T08:44:38.495 INFO:teuthology.orchestra.run.smithi100.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 7.4 MB/s | 1.5 MB 00:00 2024-03-21T08:44:38.528 INFO:teuthology.orchestra.run.smithi100.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 3.7 MB/s | 811 kB 00:00 2024-03-21T08:44:38.554 INFO:teuthology.orchestra.run.smithi100.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2024-03-21T08:44:38.612 INFO:teuthology.orchestra.run.smithi100.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 12 MB/s | 1.0 MB 00:00 2024-03-21T08:44:38.746 INFO:teuthology.orchestra.run.smithi100.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 26 MB/s | 5.7 MB 00:00 2024-03-21T08:44:38.771 INFO:teuthology.orchestra.run.smithi100.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 360 kB/s | 78 kB 00:00 2024-03-21T08:44:38.872 INFO:teuthology.orchestra.run.smithi100.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 23 MB/s | 5.8 MB 00:00 2024-03-21T08:44:38.888 INFO:teuthology.orchestra.run.smithi100.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 175 kB/s | 24 kB 00:00 2024-03-21T08:44:38.905 INFO:teuthology.orchestra.run.smithi100.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 138 kB/s | 18 kB 00:00 2024-03-21T08:44:38.930 INFO:teuthology.orchestra.run.smithi100.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 2.4 MB/s | 144 kB 00:00 2024-03-21T08:44:38.956 INFO:teuthology.orchestra.run.smithi100.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 633 kB/s | 32 kB 00:00 2024-03-21T08:44:39.006 INFO:teuthology.orchestra.run.smithi087.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-03-21T08:44:39.073 INFO:teuthology.orchestra.run.smithi100.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 4.8 MB 00:00 2024-03-21T08:44:39.098 INFO:teuthology.orchestra.run.smithi100.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.3 MB/s | 217 kB 00:00 2024-03-21T08:44:39.123 INFO:teuthology.orchestra.run.smithi100.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2024-03-21T08:44:39.149 INFO:teuthology.orchestra.run.smithi100.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 911 kB/s | 69 kB 00:00 2024-03-21T08:44:39.174 INFO:teuthology.orchestra.run.smithi100.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 814 kB/s | 41 kB 00:00 2024-03-21T08:44:39.199 INFO:teuthology.orchestra.run.smithi100.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 3.1 MB/s | 238 kB 00:00 2024-03-21T08:44:39.233 INFO:teuthology.orchestra.run.smithi100.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-21T08:44:39.258 INFO:teuthology.orchestra.run.smithi100.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 814 kB/s | 47 kB 00:00 2024-03-21T08:44:39.283 INFO:teuthology.orchestra.run.smithi100.stdout:(77/174): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2024-03-21T08:44:39.309 INFO:teuthology.orchestra.run.smithi100.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-21T08:44:39.517 INFO:teuthology.orchestra.run.smithi100.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 188 kB/s | 39 kB 00:00 2024-03-21T08:44:39.542 INFO:teuthology.orchestra.run.smithi100.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 178 kB/s | 41 kB 00:00 2024-03-21T08:44:39.701 INFO:teuthology.orchestra.run.smithi100.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-03-21T08:44:39.726 INFO:teuthology.orchestra.run.smithi100.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 300 kB 00:00 2024-03-21T08:44:39.752 INFO:teuthology.orchestra.run.smithi100.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 273 kB/s | 56 kB 00:00 2024-03-21T08:44:39.777 INFO:teuthology.orchestra.run.smithi100.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-21T08:44:39.802 INFO:teuthology.orchestra.run.smithi100.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 585 kB/s | 44 kB 00:00 2024-03-21T08:44:39.828 INFO:teuthology.orchestra.run.smithi100.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-21T08:44:39.853 INFO:teuthology.orchestra.run.smithi100.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-21T08:44:39.895 INFO:teuthology.orchestra.run.smithi100.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.1 MB 00:00 2024-03-21T08:44:39.920 INFO:teuthology.orchestra.run.smithi100.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 103 kB 00:00 2024-03-21T08:44:39.945 INFO:teuthology.orchestra.run.smithi100.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 124 kB 00:00 2024-03-21T08:44:39.971 INFO:teuthology.orchestra.run.smithi100.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.7 MB/s | 187 kB 00:00 2024-03-21T08:44:40.071 INFO:teuthology.orchestra.run.smithi100.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.2 MB/s | 187 kB 00:00 2024-03-21T08:44:40.096 INFO:teuthology.orchestra.run.smithi100.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.5 MB/s | 195 kB 00:00 2024-03-21T08:44:40.121 INFO:teuthology.orchestra.run.smithi100.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-21T08:44:40.149 INFO:teuthology.orchestra.run.smithi100.stdout:(95/174): elfutils-libelf-devel-0.190-2.el9.x86 429 kB/s | 21 kB 00:00 2024-03-21T08:44:40.164 INFO:teuthology.orchestra.run.smithi100.stdout:(96/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 609 kB/s | 25 kB 00:00 2024-03-21T08:44:40.181 INFO:teuthology.orchestra.run.smithi087.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 757 kB/s | 890 kB 00:01 2024-03-21T08:44:40.206 INFO:teuthology.orchestra.run.smithi100.stdout:(97/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.0 MB/s | 803 kB 00:00 2024-03-21T08:44:40.231 INFO:teuthology.orchestra.run.smithi100.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 3.6 MB/s | 245 kB 00:00 2024-03-21T08:44:40.256 INFO:teuthology.orchestra.run.smithi100.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-03-21T08:44:40.281 INFO:teuthology.orchestra.run.smithi100.stdout:(100/174): qatlib-23.11.0-1.el9.x86_64.rpm 4.2 MB/s | 216 kB 00:00 2024-03-21T08:44:40.307 INFO:teuthology.orchestra.run.smithi100.stdout:(101/174): qatlib-service-23.11.0-1.el9.x86_64. 692 kB/s | 35 kB 00:00 2024-03-21T08:44:40.332 INFO:teuthology.orchestra.run.smithi100.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 885 kB/s | 44 kB 00:00 2024-03-21T08:44:40.358 INFO:teuthology.orchestra.run.smithi100.stdout:(103/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 1.8 MB/s | 94 kB 00:00 2024-03-21T08:44:40.441 INFO:teuthology.orchestra.run.smithi100.stdout:(104/174): openssl-devel-3.0.7-27.el9.x86_64.rp 14 MB/s | 4.1 MB 00:00 2024-03-21T08:44:40.490 INFO:teuthology.orchestra.run.smithi087.stdout:(3/174): ceph-immutable-object-cache-19.0.0-223 471 kB/s | 145 kB 00:00 2024-03-21T08:44:40.641 INFO:teuthology.orchestra.run.smithi100.stdout:(105/174): lua-devel-5.4.4-4.el9.x86_64.rpm 72 kB/s | 22 kB 00:00 2024-03-21T08:44:40.758 INFO:teuthology.orchestra.run.smithi100.stdout:(106/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.7 MB/s | 548 kB 00:00 2024-03-21T08:44:40.817 INFO:teuthology.orchestra.run.smithi100.stdout:(107/174): grpc-data-1.46.7-10.el9.noarch.rpm 334 kB/s | 19 kB 00:00 2024-03-21T08:44:40.900 INFO:teuthology.orchestra.run.smithi100.stdout:(108/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-03-21T08:44:40.959 INFO:teuthology.orchestra.run.smithi100.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-03-21T08:44:41.017 INFO:teuthology.orchestra.run.smithi100.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-03-21T08:44:41.101 INFO:teuthology.orchestra.run.smithi100.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-03-21T08:44:41.126 INFO:teuthology.orchestra.run.smithi100.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 621 kB/s | 67 kB 00:00 2024-03-21T08:44:41.160 INFO:teuthology.orchestra.run.smithi100.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-03-21T08:44:41.227 INFO:teuthology.orchestra.run.smithi100.stdout:(114/174): python3-asyncssh-2.13.2-5.el9.noarch 8.0 MB/s | 548 kB 00:00 2024-03-21T08:44:41.261 INFO:teuthology.orchestra.run.smithi100.stdout:(115/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 6.1 MB/s | 838 kB 00:00 2024-03-21T08:44:41.294 INFO:teuthology.orchestra.run.smithi100.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 648 kB/s | 43 kB 00:00 2024-03-21T08:44:41.319 INFO:teuthology.orchestra.run.smithi100.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 551 kB/s | 32 kB 00:00 2024-03-21T08:44:41.353 INFO:teuthology.orchestra.run.smithi100.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 256 kB/s | 15 kB 00:00 2024-03-21T08:44:41.378 INFO:teuthology.orchestra.run.smithi100.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-03-21T08:44:41.420 INFO:teuthology.orchestra.run.smithi100.stdout:(120/174): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-03-21T08:44:41.445 INFO:teuthology.orchestra.run.smithi100.stdout:(121/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 4.4 MB/s | 303 kB 00:00 2024-03-21T08:44:41.479 INFO:teuthology.orchestra.run.smithi100.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 605 kB/s | 35 kB 00:00 2024-03-21T08:44:41.504 INFO:teuthology.orchestra.run.smithi100.stdout:(123/174): python3-google-auth-2.28.2-1.el9.noa 3.4 MB/s | 203 kB 00:00 2024-03-21T08:44:41.572 INFO:teuthology.orchestra.run.smithi100.stdout:(124/174): python3-grpcio-1.46.7-10.el9.x86_64. 22 MB/s | 2.0 MB 00:00 2024-03-21T08:44:41.598 INFO:teuthology.orchestra.run.smithi100.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-03-21T08:44:41.630 INFO:teuthology.orchestra.run.smithi100.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 183 kB/s | 11 kB 00:00 2024-03-21T08:44:41.656 INFO:teuthology.orchestra.run.smithi100.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 301 kB/s | 18 kB 00:00 2024-03-21T08:44:41.689 INFO:teuthology.orchestra.run.smithi100.stdout:(128/174): python3-jaraco-collections-3.0.0-8.e 396 kB/s | 23 kB 00:00 2024-03-21T08:44:41.714 INFO:teuthology.orchestra.run.smithi100.stdout:(129/174): python3-jaraco-functools-3.5.0-2.el9 332 kB/s | 19 kB 00:00 2024-03-21T08:44:41.748 INFO:teuthology.orchestra.run.smithi100.stdout:(130/174): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-03-21T08:44:41.799 INFO:teuthology.orchestra.run.smithi100.stdout:(131/174): python3-kubernetes-26.1.0-2.el9.noar 12 MB/s | 1.0 MB 00:00 2024-03-21T08:44:41.824 INFO:teuthology.orchestra.run.smithi100.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 614 kB/s | 46 kB 00:00 2024-03-21T08:44:41.857 INFO:teuthology.orchestra.run.smithi100.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-21T08:44:41.883 INFO:teuthology.orchestra.run.smithi100.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 986 kB/s | 58 kB 00:00 2024-03-21T08:44:41.917 INFO:teuthology.orchestra.run.smithi087.stdout:(4/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 1.6 MB/s | 5.4 MB 00:03 2024-03-21T08:44:41.925 INFO:teuthology.orchestra.run.smithi100.stdout:(135/174): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-03-21T08:44:41.942 INFO:teuthology.orchestra.run.smithi100.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 281 kB/s | 16 kB 00:00 2024-03-21T08:44:41.984 INFO:teuthology.orchestra.run.smithi100.stdout:(137/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-03-21T08:44:42.001 INFO:teuthology.orchestra.run.smithi100.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 481 kB/s | 28 kB 00:00 2024-03-21T08:44:42.034 INFO:teuthology.orchestra.run.smithi100.stdout:(139/174): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-03-21T08:44:42.059 INFO:teuthology.orchestra.run.smithi100.stdout:(140/174): python3-routes-2.5.1-5.el9.noarch.rp 3.2 MB/s | 188 kB 00:00 2024-03-21T08:44:42.074 INFO:teuthology.orchestra.run.smithi087.stdout:(5/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 1.5 MB/s | 2.3 MB 00:01 2024-03-21T08:44:42.093 INFO:teuthology.orchestra.run.smithi100.stdout:(141/174): protobuf-compiler-3.14.0-13.el9.x86_ 497 kB/s | 863 kB 00:01 2024-03-21T08:44:42.118 INFO:teuthology.orchestra.run.smithi100.stdout:(142/174): python3-tempora-5.0.0-2.el9.noarch.r 610 kB/s | 36 kB 00:00 2024-03-21T08:44:42.144 INFO:teuthology.orchestra.run.smithi100.stdout:(143/174): python3-rsa-4.9-2.el9.noarch.rpm 542 kB/s | 59 kB 00:00 2024-03-21T08:44:42.177 INFO:teuthology.orchestra.run.smithi100.stdout:(144/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-03-21T08:44:42.203 INFO:teuthology.orchestra.run.smithi100.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-03-21T08:44:42.236 INFO:teuthology.orchestra.run.smithi100.stdout:(146/174): python3-werkzeug-2.0.3-3.el9.1.noarc 7.1 MB/s | 427 kB 00:00 2024-03-21T08:44:42.262 INFO:teuthology.orchestra.run.smithi100.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-03-21T08:44:42.287 INFO:teuthology.orchestra.run.smithi100.stdout:(148/174): python3-typing-extensions-4.4.0-2.el 264 kB/s | 51 kB 00:00 2024-03-21T08:44:42.307 INFO:teuthology.orchestra.run.smithi087.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 3.8 MB/s | 1.5 MB 00:00 2024-03-21T08:44:42.312 INFO:teuthology.orchestra.run.smithi100.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-03-21T08:44:42.362 INFO:teuthology.orchestra.run.smithi100.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-03-21T08:44:42.404 INFO:teuthology.orchestra.run.smithi100.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.5 MB/s | 321 kB 00:00 2024-03-21T08:44:42.604 INFO:teuthology.orchestra.run.smithi100.stdout:(152/174): NetworkManager-1.47.2-1.el9.x86_64.r 12 MB/s | 2.3 MB 00:00 2024-03-21T08:44:42.680 INFO:teuthology.orchestra.run.smithi100.stdout:(153/174): NetworkManager-libnm-1.47.2-1.el9.x8 24 MB/s | 1.8 MB 00:00 2024-03-21T08:44:42.738 INFO:teuthology.orchestra.run.smithi100.stdout:(154/174): librados2-19.0.0-2231.g2edfee68.el9. 7.5 MB/s | 3.4 MB 00:00 2024-03-21T08:44:42.764 INFO:teuthology.orchestra.run.smithi100.stdout:(155/174): NetworkManager-team-1.47.2-1.el9.x86 505 kB/s | 42 kB 00:00 2024-03-21T08:44:42.803 INFO:teuthology.orchestra.run.smithi105.stdout:lab-extras 5.0 kB/s | 1.7 kB 00:00 2024-03-21T08:44:42.806 INFO:teuthology.orchestra.run.smithi100.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 71 kB 00:00 2024-03-21T08:44:42.873 INFO:teuthology.orchestra.run.smithi100.stdout:(157/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 5.6 MB/s | 3.1 MB 00:00 2024-03-21T08:44:42.898 INFO:teuthology.orchestra.run.smithi100.stdout:(158/174): elfutils-debuginfod-client-0.190-2.e 396 kB/s | 36 kB 00:00 2024-03-21T08:44:42.940 INFO:teuthology.orchestra.run.smithi100.stdout:(159/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.2 MB/s | 248 kB 00:00 2024-03-21T08:44:42.965 INFO:teuthology.orchestra.run.smithi100.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.7 MB/s | 256 kB 00:00 2024-03-21T08:44:42.990 INFO:teuthology.orchestra.run.smithi100.stdout:(161/174): libacl-2.3.1-4.el9.x86_64.rpm 460 kB/s | 23 kB 00:00 2024-03-21T08:44:43.016 INFO:teuthology.orchestra.run.smithi100.stdout:(162/174): elfutils-libelf-0.190-2.el9.x86_64.r 1.3 MB/s | 192 kB 00:00 2024-03-21T08:44:43.041 INFO:teuthology.orchestra.run.smithi100.stdout:(163/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.4 MB/s | 108 kB 00:00 2024-03-21T08:44:43.066 INFO:teuthology.orchestra.run.smithi100.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 2.0 MB/s | 155 kB 00:00 2024-03-21T08:44:43.092 INFO:teuthology.orchestra.run.smithi100.stdout:(165/174): libsmartcols-2.37.4-18.el9.x86_64.rp 1.2 MB/s | 64 kB 00:00 2024-03-21T08:44:43.117 INFO:teuthology.orchestra.run.smithi100.stdout:(166/174): libuuid-2.37.4-18.el9.x86_64.rpm 572 kB/s | 29 kB 00:00 2024-03-21T08:44:43.142 INFO:teuthology.orchestra.run.smithi100.stdout:(167/174): libmount-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 136 kB 00:00 2024-03-21T08:44:43.217 INFO:teuthology.orchestra.run.smithi100.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 9.7 MB/s | 1.2 MB 00:00 2024-03-21T08:44:43.259 INFO:teuthology.orchestra.run.smithi100.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 15 MB/s | 2.2 MB 00:00 2024-03-21T08:44:43.309 INFO:teuthology.orchestra.run.smithi100.stdout:(170/174): util-linux-2.37.4-18.el9.x86_64.rpm 14 MB/s | 2.3 MB 00:00 2024-03-21T08:44:43.335 INFO:teuthology.orchestra.run.smithi100.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 414 kB/s | 31 kB 00:00 2024-03-21T08:44:43.368 INFO:teuthology.orchestra.run.smithi100.stdout:(172/174): util-linux-core-2.37.4-18.el9.x86_64 3.0 MB/s | 465 kB 00:00 2024-03-21T08:44:43.393 INFO:teuthology.orchestra.run.smithi100.stdout:(173/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 74 kB 00:00 2024-03-21T08:44:43.453 INFO:teuthology.orchestra.run.smithi100.stdout:(174/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 7.5 MB/s | 1.1 MB 00:00 2024-03-21T08:44:43.457 INFO:teuthology.orchestra.run.smithi100.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:44:43.457 INFO:teuthology.orchestra.run.smithi100.stdout:Total 20 MB/s | 224 MB 00:11 2024-03-21T08:44:43.935 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction check 2024-03-21T08:44:43.974 INFO:teuthology.orchestra.run.smithi087.stdout:(7/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 2.5 MB/s | 4.7 MB 00:01 2024-03-21T08:44:44.206 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction check succeeded. 2024-03-21T08:44:44.207 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction test 2024-03-21T08:44:45.175 INFO:teuthology.orchestra.run.smithi105.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:44:45.176 INFO:teuthology.orchestra.run.smithi105.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-03-21T08:44:45.190 INFO:teuthology.orchestra.run.smithi105.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2024-03-21T08:44:45.191 INFO:teuthology.orchestra.run.smithi105.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-03-21T08:44:45.192 INFO:teuthology.orchestra.run.smithi105.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2024-03-21T08:44:45.193 INFO:teuthology.orchestra.run.smithi105.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2024-03-21T08:44:45.195 INFO:teuthology.orchestra.run.smithi105.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-03-21T08:44:45.198 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2024-03-21T08:44:45.199 INFO:teuthology.orchestra.run.smithi087.stdout:(8/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x86 5.8 MB/s | 17 MB 00:02 2024-03-21T08:44:45.199 INFO:teuthology.orchestra.run.smithi105.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2024-03-21T08:44:45.249 INFO:teuthology.orchestra.run.smithi087.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 505 kB/s | 25 kB 00:00 2024-03-21T08:44:45.279 INFO:teuthology.orchestra.run.smithi105.stdout:Dependencies resolved. 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: Package Arch Version Repository Size 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout:Installing: 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: ceph x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 6.6 k 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.4 M 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 890 k 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 145 k 2024-03-21T08:44:45.299 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 1.5 M 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 140 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 1.8 M 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 7.4 M 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 49 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 9.9 M 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 48 M 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 264 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 750 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 220 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 27 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 18 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2024-03-21T08:44:45.300 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 31 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 747 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 127 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 161 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 320 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 297 k 2024-03-21T08:44:45.301 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 99 k 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 85 k 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 171 k 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrading: 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager x86_64 1:1.47.2-1.el9 baseos 2.3 M 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm x86_64 1:1.47.2-1.el9 baseos 1.8 M 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team x86_64 1:1.47.2-1.el9 baseos 42 k 2024-03-21T08:44:45.302 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui x86_64 1:1.47.2-1.el9 baseos 248 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: acl x86_64 2.3.1-4.el9 baseos 71 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client x86_64 0.190-2.el9 baseos 36 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf x86_64 0.190-2.el9 baseos 192 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs x86_64 0.190-2.el9 baseos 256 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: libacl x86_64 2.3.1-4.el9 baseos 23 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel x86_64 2.3.1-4.el9 appstream 74 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid x86_64 2.37.4-18.el9 baseos 108 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk x86_64 2.37.4-18.el9 baseos 155 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: libmount x86_64 2.37.4-18.el9 baseos 136 k 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.4 M 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 3.1 M 2024-03-21T08:44:45.303 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols x86_64 2.37.4-18.el9 baseos 64 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid x86_64 2.37.4-18.el9 baseos 29 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: openssl x86_64 1:3.0.7-27.el9 baseos 1.2 M 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs x86_64 1:3.0.7-27.el9 baseos 2.2 M 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux x86_64 2.37.4-18.el9 baseos 2.3 M 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core x86_64 2.37.4-18.el9 baseos 465 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user x86_64 2.37.4-18.el9 baseos 31 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs x86_64 6.3.0-1.el9 baseos 1.1 M 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout:Installing dependencies: 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 21 M 2024-03-21T08:44:45.304 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 34 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 2.3 M 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 248 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 4.7 M 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 17 M 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2231.g2edfee68.el9 ceph-noarch 16 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 25 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-03-21T08:44:45.305 INFO:teuthology.orchestra.run.smithi105.stdout: jq x86_64 1.6-16.el9 baseos 187 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 163 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 501 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2 x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 5.1 M 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-03-21T08:44:45.306 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: lua x86_64 5.4.4-4.el9 CentOS-AppStream 187 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf x86_64 3.14.0-13.el9 CentOS-AppStream 1.0 M 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2024-03-21T08:44:45.307 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 45 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common x86_64 2:19.0.0-2231.g2edfee68.el9 ceph 131 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-03-21T08:44:45.308 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2024-03-21T08:44:45.309 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging noarch 20.9-5.el9 CentOS-AppStream 69 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf noarch 3.14.0-13.el9 CentOS-AppStream 238 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2024-03-21T08:44:45.310 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3 noarch 1.26.5-5.el9 CentOS-BaseOS 187 k 2024-03-21T08:44:45.311 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib x86_64 23.11.0-1.el9 appstream 216 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs x86_64 1.1.2-2.el9 appstream 44 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout:Installing weak dependencies: 2024-03-21T08:44:45.312 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service x86_64 23.11.0-1.el9 appstream 35 k 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction Summary 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================================ 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout:Install 150 Packages 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout:Upgrade 24 Packages 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:44:45.313 INFO:teuthology.orchestra.run.smithi105.stdout:Total download size: 224 M 2024-03-21T08:44:45.314 INFO:teuthology.orchestra.run.smithi105.stdout:Downloading Packages: 2024-03-21T08:44:45.808 INFO:teuthology.orchestra.run.smithi087.stdout:(10/174): ceph-common-19.0.0-2231.g2edfee68.el9 2.9 MB/s | 21 MB 00:07 2024-03-21T08:44:45.891 INFO:teuthology.orchestra.run.smithi087.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 2.6 MB/s | 220 kB 00:00 2024-03-21T08:44:45.942 INFO:teuthology.orchestra.run.smithi087.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 540 kB/s | 27 kB 00:00 2024-03-21T08:44:45.952 INFO:teuthology.orchestra.run.smithi100.stdout:Transaction test succeeded. 2024-03-21T08:44:45.952 INFO:teuthology.orchestra.run.smithi100.stdout:Running transaction 2024-03-21T08:44:45.995 INFO:teuthology.orchestra.run.smithi105.stdout:(1/174): ceph-19.0.0-2231.g2edfee68.el9.x86_64. 31 kB/s | 6.6 kB 00:00 2024-03-21T08:44:46.109 INFO:teuthology.orchestra.run.smithi087.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 188 kB/s | 31 kB 00:00 2024-03-21T08:44:46.475 INFO:teuthology.orchestra.run.smithi087.stdout:(14/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 2.0 MB/s | 747 kB 00:00 2024-03-21T08:44:46.537 INFO:teuthology.orchestra.run.smithi105.stdout:(2/174): ceph-fuse-19.0.0-2231.g2edfee68.el9.x8 1.6 MB/s | 890 kB 00:00 2024-03-21T08:44:46.559 INFO:teuthology.orchestra.run.smithi087.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 1.9 MB/s | 163 kB 00:00 2024-03-21T08:44:46.612 INFO:teuthology.orchestra.run.smithi105.stdout:(3/174): ceph-base-19.0.0-2231.g2edfee68.el9.x8 6.5 MB/s | 5.4 MB 00:00 2024-03-21T08:44:46.626 INFO:teuthology.orchestra.run.smithi087.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.9 MB/s | 127 kB 00:00 2024-03-21T08:44:46.679 INFO:teuthology.orchestra.run.smithi105.stdout:(4/174): ceph-immutable-object-cache-19.0.0-223 1.0 MB/s | 145 kB 00:00 2024-03-21T08:44:46.726 INFO:teuthology.orchestra.run.smithi087.stdout:(17/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 3.6 MB/s | 9.9 MB 00:02 2024-03-21T08:44:46.910 INFO:teuthology.orchestra.run.smithi087.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 1.7 MB/s | 501 kB 00:00 2024-03-21T08:44:46.938 INFO:teuthology.orchestra.run.smithi105.stdout:(5/174): ceph-mds-19.0.0-2231.g2edfee68.el9.x86 7.1 MB/s | 2.3 MB 00:00 2024-03-21T08:44:46.960 INFO:teuthology.orchestra.run.smithi087.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 904 kB/s | 45 kB 00:00 2024-03-21T08:44:47.035 INFO:teuthology.orchestra.run.smithi087.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.7 MB/s | 131 kB 00:00 2024-03-21T08:44:47.111 INFO:teuthology.orchestra.run.smithi087.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 2.1 MB/s | 161 kB 00:00 2024-03-21T08:44:47.244 INFO:teuthology.orchestra.run.smithi087.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 2.3 MB/s | 320 kB 00:00 2024-03-21T08:44:47.255 INFO:teuthology.orchestra.run.smithi105.stdout:(6/174): ceph-mgr-19.0.0-2231.g2edfee68.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-03-21T08:44:47.370 INFO:teuthology.orchestra.run.smithi087.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 2.3 MB/s | 297 kB 00:00 2024-03-21T08:44:47.454 INFO:teuthology.orchestra.run.smithi087.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.2 MB/s | 99 kB 00:00 2024-03-21T08:44:47.537 INFO:teuthology.orchestra.run.smithi087.stdout:(25/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.0 MB/s | 85 kB 00:00 2024-03-21T08:44:47.822 INFO:teuthology.orchestra.run.smithi105.stdout:(7/174): ceph-common-19.0.0-2231.g2edfee68.el9. 10 MB/s | 21 MB 00:02 2024-03-21T08:44:47.980 INFO:teuthology.orchestra.run.smithi105.stdout:(8/174): ceph-mon-19.0.0-2231.g2edfee68.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-03-21T08:44:48.055 INFO:teuthology.orchestra.run.smithi105.stdout:(9/174): ceph-selinux-19.0.0-2231.g2edfee68.el9 336 kB/s | 25 kB 00:00 2024-03-21T08:44:48.371 INFO:teuthology.orchestra.run.smithi087.stdout:(26/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 3.7 MB/s | 3.1 MB 00:00 2024-03-21T08:44:48.471 INFO:teuthology.orchestra.run.smithi087.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 1.7 MB/s | 171 kB 00:00 2024-03-21T08:44:48.563 INFO:teuthology.orchestra.run.smithi087.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 370 kB/s | 34 kB 00:00 2024-03-21T08:44:48.663 INFO:teuthology.orchestra.run.smithi087.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.4 MB/s | 140 kB 00:00 2024-03-21T08:44:48.964 INFO:teuthology.orchestra.run.smithi105.stdout:(10/174): ceph-radosgw-19.0.0-2231.g2edfee68.el 8.6 MB/s | 9.9 MB 00:01 2024-03-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi105.stdout:(11/174): cephfs-mirror-19.0.0-2231.g2edfee68.e 980 kB/s | 220 kB 00:00 2024-03-21T08:44:49.281 INFO:teuthology.orchestra.run.smithi105.stdout:(12/174): cephfs-shell-19.0.0-2231.g2edfee68.el 295 kB/s | 27 kB 00:00 2024-03-21T08:44:49.373 INFO:teuthology.orchestra.run.smithi105.stdout:(13/174): libcephfs-devel-19.0.0-2231.g2edfee68 342 kB/s | 31 kB 00:00 2024-03-21T08:44:49.471 INFO:teuthology.orchestra.run.smithi087.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 2.2 MB/s | 1.8 MB 00:00 2024-03-21T08:44:49.548 INFO:teuthology.orchestra.run.smithi105.stdout:(14/174): ceph-osd-19.0.0-2231.g2edfee68.el9.x8 7.3 MB/s | 17 MB 00:02 2024-03-21T08:44:49.597 INFO:teuthology.orchestra.run.smithi087.stdout:(31/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 1.8 MB/s | 5.1 MB 00:02 2024-03-21T08:44:49.615 INFO:teuthology.orchestra.run.smithi105.stdout:(15/174): libcephsqlite-19.0.0-2231.g2edfee68.e 2.4 MB/s | 163 kB 00:00 2024-03-21T08:44:49.691 INFO:teuthology.orchestra.run.smithi105.stdout:(16/174): librados-devel-19.0.0-2231.g2edfee68. 1.6 MB/s | 127 kB 00:00 2024-03-21T08:44:49.697 INFO:teuthology.orchestra.run.smithi087.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.4 MB/s | 248 kB 00:00 2024-03-21T08:44:49.716 INFO:teuthology.orchestra.run.smithi105.stdout:(17/174): libcephfs2-19.0.0-2231.g2edfee68.el9. 2.1 MB/s | 747 kB 00:00 2024-03-21T08:44:49.764 INFO:teuthology.orchestra.run.smithi087.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 741 kB/s | 49 kB 00:00 2024-03-21T08:44:49.766 INFO:teuthology.orchestra.run.smithi105.stdout:(18/174): libradosstriper1-19.0.0-2231.g2edfee6 6.5 MB/s | 501 kB 00:00 2024-03-21T08:44:49.806 INFO:teuthology.orchestra.run.smithi087.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 397 kB/s | 16 kB 00:00 2024-03-21T08:44:49.841 INFO:teuthology.orchestra.run.smithi105.stdout:(19/174): python3-ceph-argparse-19.0.0-2231.g2e 601 kB/s | 45 kB 00:00 2024-03-21T08:44:49.917 INFO:teuthology.orchestra.run.smithi105.stdout:(20/174): python3-ceph-common-19.0.0-2231.g2edf 1.7 MB/s | 131 kB 00:00 2024-03-21T08:44:49.923 INFO:teuthology.orchestra.run.smithi087.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.2 MB/s | 264 kB 00:00 2024-03-21T08:44:49.984 INFO:teuthology.orchestra.run.smithi105.stdout:(21/174): python3-cephfs-19.0.0-2231.g2edfee68. 2.4 MB/s | 161 kB 00:00 2024-03-21T08:44:50.067 INFO:teuthology.orchestra.run.smithi105.stdout:(22/174): python3-rados-19.0.0-2231.g2edfee68.e 3.8 MB/s | 320 kB 00:00 2024-03-21T08:44:50.151 INFO:teuthology.orchestra.run.smithi105.stdout:(23/174): python3-rbd-19.0.0-2231.g2edfee68.el9 3.5 MB/s | 297 kB 00:00 2024-03-21T08:44:50.215 INFO:teuthology.orchestra.run.smithi087.stdout:(36/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 2.5 MB/s | 750 kB 00:00 2024-03-21T08:44:50.234 INFO:teuthology.orchestra.run.smithi105.stdout:(24/174): python3-rgw-19.0.0-2231.g2edfee68.el9 1.2 MB/s | 99 kB 00:00 2024-03-21T08:44:50.273 INFO:teuthology.orchestra.run.smithi087.stdout:(37/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 312 kB/s | 18 kB 00:00 2024-03-21T08:44:50.301 INFO:teuthology.orchestra.run.smithi105.stdout:(25/174): rbd-fuse-19.0.0-2231.g2edfee68.el9.x8 1.3 MB/s | 85 kB 00:00 2024-03-21T08:44:50.407 INFO:teuthology.orchestra.run.smithi087.stdout:(38/174): automake-1.16.2-8.el9.noarch.rpm 4.9 MB/s | 662 kB 00:00 2024-03-21T08:44:50.474 INFO:teuthology.orchestra.run.smithi087.stdout:(39/174): bison-3.7.4-5.el9.x86_64.rpm 14 MB/s | 920 kB 00:00 2024-03-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi087.stdout:(40/174): boost-program-options-1.75.0-8.el9.x8 3.1 MB/s | 106 kB 00:00 2024-03-21T08:44:50.567 INFO:teuthology.orchestra.run.smithi087.stdout:(41/174): flex-2.6.4-9.el9.x86_64.rpm 5.1 MB/s | 307 kB 00:00 2024-03-21T08:44:50.600 INFO:teuthology.orchestra.run.smithi087.stdout:(42/174): flexiblas-3.0.4-8.el9.x86_64.rpm 953 kB/s | 32 kB 00:00 2024-03-21T08:44:50.602 INFO:teuthology.orchestra.run.smithi105.stdout:(26/174): rbd-mirror-19.0.0-2231.g2edfee68.el9. 10 MB/s | 3.1 MB 00:00 2024-03-21T08:44:50.685 INFO:teuthology.orchestra.run.smithi105.stdout:(27/174): rbd-nbd-19.0.0-2231.g2edfee68.el9.x86 2.0 MB/s | 171 kB 00:00 2024-03-21T08:44:50.708 INFO:teuthology.orchestra.run.smithi087.stdout:(43/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 28 MB/s | 3.0 MB 00:00 2024-03-21T08:44:50.742 INFO:teuthology.orchestra.run.smithi087.stdout:(44/174): flexiblas-openblas-openmp-3.0.4-8.el9 516 kB/s | 17 kB 00:00 2024-03-21T08:44:50.752 INFO:teuthology.orchestra.run.smithi105.stdout:(28/174): ceph-grafana-dashboards-19.0.0-2231.g 509 kB/s | 34 kB 00:00 2024-03-21T08:44:50.776 INFO:teuthology.orchestra.run.smithi087.stdout:(45/174): libaio-devel-0.3.111-13.el9.x86_64.rp 301 kB/s | 10 kB 00:00 2024-03-21T08:44:50.809 INFO:teuthology.orchestra.run.smithi087.stdout:(46/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.7 MB/s | 159 kB 00:00 2024-03-21T08:44:50.843 INFO:teuthology.orchestra.run.smithi087.stdout:(47/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 44 kB 00:00 2024-03-21T08:44:50.844 INFO:teuthology.orchestra.run.smithi105.stdout:(29/174): ceph-mgr-cephadm-19.0.0-2231.g2edfee6 1.5 MB/s | 140 kB 00:00 2024-03-21T08:44:50.885 INFO:teuthology.orchestra.run.smithi087.stdout:(48/174): librdkafka-1.6.1-102.el9.x86_64.rpm 16 MB/s | 661 kB 00:00 2024-03-21T08:44:50.935 INFO:teuthology.orchestra.run.smithi087.stdout:(49/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-21T08:44:50.969 INFO:teuthology.orchestra.run.smithi087.stdout:(50/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 46 kB 00:00 2024-03-21T08:44:51.011 INFO:teuthology.orchestra.run.smithi087.stdout:(51/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 6.7 MB/s | 282 kB 00:00 2024-03-21T08:44:51.045 INFO:teuthology.orchestra.run.smithi087.stdout:(52/174): lua-5.4.4-4.el9.x86_64.rpm 5.4 MB/s | 187 kB 00:00 2024-03-21T08:44:51.078 INFO:teuthology.orchestra.run.smithi087.stdout:(53/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 6.4 MB/s | 217 kB 00:00 2024-03-21T08:44:51.086 INFO:teuthology.orchestra.run.smithi105.stdout:(30/174): ceph-mgr-dashboard-19.0.0-2231.g2edfe 7.4 MB/s | 1.8 MB 00:00 2024-03-21T08:44:51.237 INFO:teuthology.orchestra.run.smithi105.stdout:(31/174): librgw2-19.0.0-2231.g2edfee68.el9.x86 3.4 MB/s | 5.1 MB 00:01 2024-03-21T08:44:51.238 INFO:teuthology.orchestra.run.smithi087.stdout:(54/174): openblas-0.3.21-2.el9.x86_64.rpm 215 kB/s | 34 kB 00:00 2024-03-21T08:44:51.337 INFO:teuthology.orchestra.run.smithi105.stdout:(32/174): ceph-mgr-modules-core-19.0.0-2231.g2e 2.4 MB/s | 248 kB 00:00 2024-03-21T08:44:51.388 INFO:teuthology.orchestra.run.smithi087.stdout:(55/174): openblas-openmp-0.3.21-2.el9.x86_64.r 32 MB/s | 4.7 MB 00:00 2024-03-21T08:44:51.420 INFO:teuthology.orchestra.run.smithi105.stdout:(33/174): ceph-mgr-rook-19.0.0-2231.g2edfee68.e 592 kB/s | 49 kB 00:00 2024-03-21T08:44:51.454 INFO:teuthology.orchestra.run.smithi087.stdout:(56/174): postgresql-13.11-1.el9.x86_64.rpm 23 MB/s | 1.5 MB 00:00 2024-03-21T08:44:51.496 INFO:teuthology.orchestra.run.smithi105.stdout:(34/174): ceph-prometheus-alerts-19.0.0-2231.g2 220 kB/s | 16 kB 00:00 2024-03-21T08:44:51.513 INFO:teuthology.orchestra.run.smithi087.stdout:(57/174): postgresql-contrib-13.11-1.el9.x86_64 14 MB/s | 811 kB 00:00 2024-03-21T08:44:51.547 INFO:teuthology.orchestra.run.smithi087.stdout:(58/174): postgresql-private-libs-13.11-1.el9.x 4.0 MB/s | 134 kB 00:00 2024-03-21T08:44:51.613 INFO:teuthology.orchestra.run.smithi105.stdout:(35/174): ceph-volume-19.0.0-2231.g2edfee68.el9 2.2 MB/s | 264 kB 00:00 2024-03-21T08:44:51.714 INFO:teuthology.orchestra.run.smithi087.stdout:(59/174): postgresql-server-13.11-1.el9.x86_64. 34 MB/s | 5.7 MB 00:00 2024-03-21T08:44:51.763 INFO:teuthology.orchestra.run.smithi105.stdout:(36/174): ceph-mgr-diskprediction-local-19.0.0- 11 MB/s | 7.4 MB 00:00 2024-03-21T08:44:51.764 INFO:teuthology.orchestra.run.smithi087.stdout:(60/174): protobuf-3.14.0-13.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2024-03-21T08:44:51.798 INFO:teuthology.orchestra.run.smithi087.stdout:(61/174): python3-attrs-20.3.0-7.el9.noarch.rpm 2.3 MB/s | 78 kB 00:00 2024-03-21T08:44:51.830 INFO:teuthology.orchestra.run.smithi105.stdout:(37/174): cephadm-19.0.0-2231.g2edfee68.el9.noa 3.4 MB/s | 750 kB 00:00 2024-03-21T08:44:51.847 INFO:teuthology.orchestra.run.smithi105.stdout:(38/174): cephfs-top-19.0.0-2231.g2edfee68.el9. 219 kB/s | 18 kB 00:00 2024-03-21T08:44:51.948 INFO:teuthology.orchestra.run.smithi087.stdout:(62/174): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-03-21T08:44:52.023 INFO:teuthology.orchestra.run.smithi087.stdout:(63/174): python3-babel-2.9.1-2.el9.noarch.rpm 26 MB/s | 5.8 MB 00:00 2024-03-21T08:44:52.039 INFO:teuthology.orchestra.run.smithi105.stdout:(39/174): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 662 kB 00:00 2024-03-21T08:44:52.040 INFO:teuthology.orchestra.run.smithi087.stdout:(64/174): python3-jsonpatch-1.21-16.el9.noarch. 272 kB/s | 24 kB 00:00 2024-03-21T08:44:52.058 INFO:teuthology.orchestra.run.smithi087.stdout:(65/174): python3-jsonpointer-2.0-4.el9.noarch. 543 kB/s | 18 kB 00:00 2024-03-21T08:44:52.081 INFO:teuthology.orchestra.run.smithi105.stdout:(40/174): bison-3.7.4-5.el9.x86_64.rpm 3.8 MB/s | 920 kB 00:00 2024-03-21T08:44:52.082 INFO:teuthology.orchestra.run.smithi087.stdout:(66/174): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 144 kB 00:00 2024-03-21T08:44:52.106 INFO:teuthology.orchestra.run.smithi105.stdout:(41/174): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 106 kB 00:00 2024-03-21T08:44:52.107 INFO:teuthology.orchestra.run.smithi087.stdout:(67/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-21T08:44:52.131 INFO:teuthology.orchestra.run.smithi105.stdout:(42/174): flex-2.6.4-9.el9.x86_64.rpm 6.0 MB/s | 307 kB 00:00 2024-03-21T08:44:52.156 INFO:teuthology.orchestra.run.smithi105.stdout:(43/174): flexiblas-3.0.4-8.el9.x86_64.rpm 637 kB/s | 32 kB 00:00 2024-03-21T08:44:52.225 INFO:teuthology.orchestra.run.smithi087.stdout:(68/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 34 MB/s | 4.8 MB 00:00 2024-03-21T08:44:52.248 INFO:teuthology.orchestra.run.smithi105.stdout:(44/174): flexiblas-netlib-3.0.4-8.el9.x86_64.r 26 MB/s | 3.0 MB 00:00 2024-03-21T08:44:52.250 INFO:teuthology.orchestra.run.smithi087.stdout:(69/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2024-03-21T08:44:52.274 INFO:teuthology.orchestra.run.smithi105.stdout:(45/174): flexiblas-openblas-openmp-3.0.4-8.el9 148 kB/s | 17 kB 00:00 2024-03-21T08:44:52.275 INFO:teuthology.orchestra.run.smithi087.stdout:(70/174): python3-oauthlib-3.1.1-5.el9.noarch.r 3.4 MB/s | 174 kB 00:00 2024-03-21T08:44:52.299 INFO:teuthology.orchestra.run.smithi105.stdout:(46/174): libaio-devel-0.3.111-13.el9.x86_64.rp 201 kB/s | 10 kB 00:00 2024-03-21T08:44:52.300 INFO:teuthology.orchestra.run.smithi087.stdout:(71/174): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 69 kB 00:00 2024-03-21T08:44:52.324 INFO:teuthology.orchestra.run.smithi105.stdout:(47/174): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 159 kB 00:00 2024-03-21T08:44:52.326 INFO:teuthology.orchestra.run.smithi087.stdout:(72/174): python3-prettytable-0.7.2-27.el9.noar 824 kB/s | 41 kB 00:00 2024-03-21T08:44:52.349 INFO:teuthology.orchestra.run.smithi105.stdout:(48/174): librabbitmq-0.11.0-7.el9.x86_64.rpm 880 kB/s | 44 kB 00:00 2024-03-21T08:44:52.351 INFO:teuthology.orchestra.run.smithi087.stdout:(73/174): python3-protobuf-3.14.0-13.el9.noarch 4.6 MB/s | 238 kB 00:00 2024-03-21T08:44:52.375 INFO:teuthology.orchestra.run.smithi105.stdout:(49/174): librdkafka-1.6.1-102.el9.x86_64.rpm 13 MB/s | 661 kB 00:00 2024-03-21T08:44:52.376 INFO:teuthology.orchestra.run.smithi087.stdout:(74/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 132 kB 00:00 2024-03-21T08:44:52.400 INFO:teuthology.orchestra.run.smithi105.stdout:(50/174): libtool-2.4.6-45.el9.x86_64.rpm 11 MB/s | 578 kB 00:00 2024-03-21T08:44:52.401 INFO:teuthology.orchestra.run.smithi087.stdout:(75/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-21T08:44:52.427 INFO:teuthology.orchestra.run.smithi087.stdout:(76/174): python3-pytz-2021.1-5.el9.noarch.rpm 949 kB/s | 47 kB 00:00 2024-03-21T08:44:52.438 INFO:teuthology.orchestra.run.smithi105.stdout:(51/174): libzstd-devel-1.5.1-2.el9.x86_64.rpm 747 kB/s | 46 kB 00:00 2024-03-21T08:44:52.452 INFO:teuthology.orchestra.run.smithi087.stdout:(77/174): python3-requests-oauthlib-1.3.0-12.el 855 kB/s | 43 kB 00:00 2024-03-21T08:44:52.464 INFO:teuthology.orchestra.run.smithi105.stdout:(52/174): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.4 MB/s | 282 kB 00:00 2024-03-21T08:44:52.489 INFO:teuthology.orchestra.run.smithi105.stdout:(53/174): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 187 kB 00:00 2024-03-21T08:44:52.514 INFO:teuthology.orchestra.run.smithi105.stdout:(54/174): oniguruma-6.9.6-1.el9.5.x86_64.rpm 4.3 MB/s | 217 kB 00:00 2024-03-21T08:44:52.531 INFO:teuthology.orchestra.run.smithi105.stdout:(55/174): openblas-0.3.21-2.el9.x86_64.rpm 820 kB/s | 34 kB 00:00 2024-03-21T08:44:52.619 INFO:teuthology.orchestra.run.smithi087.stdout:(78/174): python3-toml-0.10.2-6.el9.noarch.rpm 235 kB/s | 39 kB 00:00 2024-03-21T08:44:52.811 INFO:teuthology.orchestra.run.smithi087.stdout:(79/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-03-21T08:44:52.836 INFO:teuthology.orchestra.run.smithi087.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 190 kB/s | 41 kB 00:00 2024-03-21T08:44:52.858 INFO:teuthology.orchestra.run.smithi100.stdout: Preparing : 1/1 2024-03-21T08:44:52.862 INFO:teuthology.orchestra.run.smithi087.stdout:(81/174): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 300 kB 00:00 2024-03-21T08:44:52.887 INFO:teuthology.orchestra.run.smithi087.stdout:(82/174): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 56 kB 00:00 2024-03-21T08:44:52.912 INFO:teuthology.orchestra.run.smithi087.stdout:(83/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-21T08:44:52.938 INFO:teuthology.orchestra.run.smithi087.stdout:(84/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 882 kB/s | 44 kB 00:00 2024-03-21T08:44:52.963 INFO:teuthology.orchestra.run.smithi087.stdout:(85/174): fuse-2.9.9-15.el9.x86_64.rpm 1.5 MB/s | 79 kB 00:00 2024-03-21T08:44:52.975 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:44:52.988 INFO:teuthology.orchestra.run.smithi087.stdout:(86/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.7 MB/s | 241 kB 00:00 2024-03-21T08:44:53.022 INFO:teuthology.orchestra.run.smithi087.stdout:(87/174): python3-cryptography-36.0.1-4.el9.x86 20 MB/s | 1.1 MB 00:00 2024-03-21T08:44:53.041 INFO:teuthology.orchestra.run.smithi105.stdout:(56/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 9.6 MB/s | 48 MB 00:04 2024-03-21T08:44:53.047 INFO:teuthology.orchestra.run.smithi087.stdout:(88/174): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 103 kB 00:00 2024-03-21T08:44:53.073 INFO:teuthology.orchestra.run.smithi087.stdout:(89/174): python3-pycparser-2.20-6.el9.noarch.r 2.4 MB/s | 124 kB 00:00 2024-03-21T08:44:53.098 INFO:teuthology.orchestra.run.smithi087.stdout:(90/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-21T08:44:53.107 INFO:teuthology.orchestra.run.smithi105.stdout:(57/174): openblas-openmp-0.3.21-2.el9.x86_64.r 8.0 MB/s | 4.7 MB 00:00 2024-03-21T08:44:53.141 INFO:teuthology.orchestra.run.smithi105.stdout:(58/174): postgresql-13.11-1.el9.x86_64.rpm 2.5 MB/s | 1.5 MB 00:00 2024-03-21T08:44:53.166 INFO:teuthology.orchestra.run.smithi105.stdout:(59/174): postgresql-contrib-13.11-1.el9.x86_64 6.5 MB/s | 811 kB 00:00 2024-03-21T08:44:53.183 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:44:53.191 INFO:teuthology.orchestra.run.smithi105.stdout:(60/174): postgresql-private-libs-13.11-1.el9.x 1.6 MB/s | 134 kB 00:00 2024-03-21T08:44:53.257 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:44:53.268 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:44:53.292 INFO:teuthology.orchestra.run.smithi105.stdout:(61/174): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-03-21T08:44:53.317 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:44:53.332 INFO:teuthology.orchestra.run.smithi087.stdout:(91/174): jq-1.6-16.el9.x86_64.rpm 725 kB/s | 187 kB 00:00 2024-03-21T08:44:53.375 INFO:teuthology.orchestra.run.smithi105.stdout:(62/174): postgresql-server-13.11-1.el9.x86_64. 27 MB/s | 5.7 MB 00:00 2024-03-21T08:44:53.379 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:44:53.400 INFO:teuthology.orchestra.run.smithi105.stdout:(63/174): python3-attrs-20.3.0-7.el9.noarch.rpm 374 kB/s | 78 kB 00:00 2024-03-21T08:44:53.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:44:53.444 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:44:53.488 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:44:53.498 INFO:teuthology.orchestra.run.smithi087.stdout:(92/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 195 kB 00:00 2024-03-21T08:44:53.509 INFO:teuthology.orchestra.run.smithi105.stdout:(64/174): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 5.8 MB 00:00 2024-03-21T08:44:53.534 INFO:teuthology.orchestra.run.smithi105.stdout:(65/174): python3-jsonpatch-1.21-16.el9.noarch. 186 kB/s | 24 kB 00:00 2024-03-21T08:44:53.560 INFO:teuthology.orchestra.run.smithi105.stdout:(66/174): python3-jsonpointer-2.0-4.el9.noarch. 116 kB/s | 18 kB 00:00 2024-03-21T08:44:53.585 INFO:teuthology.orchestra.run.smithi105.stdout:(67/174): python3-mako-1.1.4-6.el9.noarch.rpm 1.9 MB/s | 144 kB 00:00 2024-03-21T08:44:53.610 INFO:teuthology.orchestra.run.smithi105.stdout:(68/174): python3-markupsafe-1.1.1-12.el9.x86_6 632 kB/s | 32 kB 00:00 2024-03-21T08:44:53.615 INFO:teuthology.orchestra.run.smithi087.stdout:(93/174): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-03-21T08:44:53.752 INFO:teuthology.orchestra.run.smithi105.stdout:(69/174): python3-numpy-1.20.1-5.el9.x86_64.rpm 25 MB/s | 4.8 MB 00:00 2024-03-21T08:44:53.774 INFO:teuthology.orchestra.run.smithi087.stdout:(94/174): libgfortran-11.4.1-3.el9.x86_64.rpm 1.2 MB/s | 803 kB 00:00 2024-03-21T08:44:53.778 INFO:teuthology.orchestra.run.smithi105.stdout:(70/174): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 217 kB 00:00 2024-03-21T08:44:53.802 INFO:teuthology.orchestra.run.smithi105.stdout:(71/174): python3-oauthlib-3.1.1-5.el9.noarch.r 921 kB/s | 174 kB 00:00 2024-03-21T08:44:53.815 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:44:53.828 INFO:teuthology.orchestra.run.smithi105.stdout:(72/174): python3-packaging-20.9-5.el9.noarch.r 917 kB/s | 69 kB 00:00 2024-03-21T08:44:53.853 INFO:teuthology.orchestra.run.smithi105.stdout:(73/174): python3-prettytable-0.7.2-27.el9.noar 821 kB/s | 41 kB 00:00 2024-03-21T08:44:53.886 INFO:teuthology.orchestra.run.smithi105.stdout:(74/174): python3-protobuf-3.14.0-13.el9.noarch 2.8 MB/s | 238 kB 00:00 2024-03-21T08:44:53.912 INFO:teuthology.orchestra.run.smithi105.stdout:(75/174): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 132 kB 00:00 2024-03-21T08:44:53.920 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:44:53.937 INFO:teuthology.orchestra.run.smithi105.stdout:(76/174): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 211 kB 00:00 2024-03-21T08:44:53.962 INFO:teuthology.orchestra.run.smithi105.stdout:(77/174): python3-pytz-2021.1-5.el9.noarch.rpm 629 kB/s | 47 kB 00:00 2024-03-21T08:44:53.987 INFO:teuthology.orchestra.run.smithi105.stdout:(78/174): python3-requests-oauthlib-1.3.0-12.el 569 kB/s | 43 kB 00:00 2024-03-21T08:44:54.024 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:44:54.046 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:44:54.119 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:44:54.158 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:44:54.209 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:44:54.257 INFO:teuthology.orchestra.run.smithi105.stdout:(79/174): python3-toml-0.10.2-6.el9.noarch.rpm 146 kB/s | 39 kB 00:00 2024-03-21T08:44:54.280 INFO:teuthology.orchestra.run.smithi105.stdout:(80/174): python3-wcwidth-0.2.5-8.el9.noarch.rp 141 kB/s | 41 kB 00:00 2024-03-21T08:44:54.291 INFO:teuthology.orchestra.run.smithi087.stdout:(95/174): ceph-test-19.0.0-2231.g2edfee68.el9.x 5.3 MB/s | 48 MB 00:09 2024-03-21T08:44:54.390 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:44:54.430 INFO:teuthology.orchestra.run.smithi105.stdout:(81/174): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-03-21T08:44:54.448 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:44:54.455 INFO:teuthology.orchestra.run.smithi105.stdout:(82/174): socat-1.7.4.1-5.el9.x86_64.rpm 1.5 MB/s | 300 kB 00:00 2024-03-21T08:44:54.476 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:44:54.480 INFO:teuthology.orchestra.run.smithi105.stdout:(83/174): uuid-1.6.2-55.el9.x86_64.rpm 284 kB/s | 56 kB 00:00 2024-03-21T08:44:54.506 INFO:teuthology.orchestra.run.smithi105.stdout:(84/174): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 63 kB 00:00 2024-03-21T08:44:54.531 INFO:teuthology.orchestra.run.smithi105.stdout:(85/174): zlib-devel-1.2.11-40.el9.x86_64.rpm 589 kB/s | 44 kB 00:00 2024-03-21T08:44:54.556 INFO:teuthology.orchestra.run.smithi105.stdout:(86/174): fuse-2.9.9-15.el9.x86_64.rpm 1.0 MB/s | 79 kB 00:00 2024-03-21T08:44:54.579 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:44:54.598 INFO:teuthology.orchestra.run.smithi105.stdout:(87/174): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2024-03-21T08:44:54.602 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:44:54.631 INFO:teuthology.orchestra.run.smithi105.stdout:(88/174): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2024-03-21T08:44:54.649 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:44:54.656 INFO:teuthology.orchestra.run.smithi105.stdout:(89/174): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 103 kB 00:00 2024-03-21T08:44:54.682 INFO:teuthology.orchestra.run.smithi105.stdout:(90/174): python3-pycparser-2.20-6.el9.noarch.r 1.5 MB/s | 124 kB 00:00 2024-03-21T08:44:54.691 INFO:teuthology.orchestra.run.smithi087.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 20 kB/s | 21 kB 00:01 2024-03-21T08:44:54.698 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:44:54.709 INFO:teuthology.orchestra.run.smithi105.stdout:(91/174): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 187 kB 00:00 2024-03-21T08:44:54.721 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:44:54.736 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:44:54.833 INFO:teuthology.orchestra.run.smithi105.stdout:(92/174): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2024-03-21T08:44:54.833 INFO:teuthology.orchestra.run.smithi087.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 24 kB/s | 25 kB 00:01 2024-03-21T08:44:54.858 INFO:teuthology.orchestra.run.smithi105.stdout:(93/174): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-03-21T08:44:54.883 INFO:teuthology.orchestra.run.smithi105.stdout:(94/174): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-03-21T08:44:54.908 INFO:teuthology.orchestra.run.smithi105.stdout:(95/174): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-03-21T08:44:54.929 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:44:54.950 INFO:teuthology.orchestra.run.smithi105.stdout:(96/174): elfutils-libelf-devel-0.190-2.el9.x86 233 kB/s | 21 kB 00:00 2024-03-21T08:44:54.960 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:44:54.967 INFO:teuthology.orchestra.run.smithi105.stdout:(97/174): libuuid-devel-2.37.4-18.el9.x86_64.rp 304 kB/s | 25 kB 00:00 2024-03-21T08:44:54.992 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:44:55.067 INFO:teuthology.orchestra.run.smithi105.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 2.1 MB/s | 245 kB 00:00 2024-03-21T08:44:55.093 INFO:teuthology.orchestra.run.smithi105.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 1.9 MB/s | 248 kB 00:00 2024-03-21T08:44:55.096 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:44:55.135 INFO:teuthology.orchestra.run.smithi105.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 835 kB/s | 35 kB 00:00 2024-03-21T08:44:55.160 INFO:teuthology.orchestra.run.smithi105.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 2.3 MB/s | 216 kB 00:00 2024-03-21T08:44:55.177 INFO:teuthology.orchestra.run.smithi105.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 1.0 MB/s | 44 kB 00:00 2024-03-21T08:44:55.194 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:44:55.219 INFO:teuthology.orchestra.run.smithi105.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 534 kB/s | 22 kB 00:00 2024-03-21T08:44:55.254 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:44:55.327 INFO:teuthology.orchestra.run.smithi105.stdout:(104/174): protobuf-compiler-3.14.0-13.el9.x86_ 7.8 MB/s | 863 kB 00:00 2024-03-21T08:44:55.334 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:44:55.352 INFO:teuthology.orchestra.run.smithi105.stdout:(105/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 487 kB/s | 94 kB 00:00 2024-03-21T08:44:55.389 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:44:55.575 INFO:teuthology.orchestra.run.smithi087.stdout:(98/174): python3-devel-3.9.17-2.el9.x86_64.rpm 278 kB/s | 245 kB 00:00 2024-03-21T08:44:55.619 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:44:55.661 INFO:teuthology.orchestra.run.smithi105.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 1.0 MB/s | 309 kB 00:00 2024-03-21T08:44:55.687 INFO:teuthology.orchestra.run.smithi105.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 1.5 MB/s | 548 kB 00:00 2024-03-21T08:44:55.708 INFO:teuthology.orchestra.run.smithi087.stdout:(99/174): python3-jinja2-2.11.3-5.el9.noarch.rp 283 kB/s | 248 kB 00:00 2024-03-21T08:44:55.730 INFO:teuthology.orchestra.run.smithi105.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 286 kB/s | 19 kB 00:00 2024-03-21T08:44:55.732 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:44:55.788 INFO:teuthology.orchestra.run.smithi105.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 445 kB/s | 26 kB 00:00 2024-03-21T08:44:55.823 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:44:55.853 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:44:55.855 INFO:teuthology.orchestra.run.smithi105.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 731 kB/s | 49 kB 00:00 2024-03-21T08:44:55.888 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:44:55.922 INFO:teuthology.orchestra.run.smithi105.stdout:(111/174): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-03-21T08:44:55.940 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:44:55.989 INFO:teuthology.orchestra.run.smithi105.stdout:(112/174): luarocks-3.9.2-1.el9.noarch.rpm 2.2 MB/s | 151 kB 00:00 2024-03-21T08:44:56.013 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:44:56.056 INFO:teuthology.orchestra.run.smithi105.stdout:(113/174): libarrow-9.0.0-10.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-03-21T08:44:56.068 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:44:56.106 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:44:56.123 INFO:teuthology.orchestra.run.smithi105.stdout:(114/174): openssl-devel-3.0.7-27.el9.x86_64.rp 3.4 MB/s | 4.1 MB 00:01 2024-03-21T08:44:56.157 INFO:teuthology.orchestra.run.smithi105.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 5.3 MB/s | 548 kB 00:00 2024-03-21T08:44:56.187 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:44:56.207 INFO:teuthology.orchestra.run.smithi105.stdout:(116/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.8 MB/s | 838 kB 00:00 2024-03-21T08:44:56.222 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:44:56.241 INFO:teuthology.orchestra.run.smithi105.stdout:(117/174): python3-cachetools-4.2.4-1.el9.noarc 549 kB/s | 32 kB 00:00 2024-03-21T08:44:56.266 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:44:56.298 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:44:56.317 INFO:teuthology.orchestra.run.smithi087.stdout:(100/174): qatlib-service-23.11.0-1.el9.x86_64. 57 kB/s | 35 kB 00:00 2024-03-21T08:44:56.334 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:44:56.341 INFO:teuthology.orchestra.run.smithi105.stdout:(118/174): python3-certifi-2021.10.8-2.el9.noar 112 kB/s | 15 kB 00:00 2024-03-21T08:44:56.391 INFO:teuthology.orchestra.run.smithi105.stdout:(119/174): python3-cheroot-8.6.0-4.el9.noarch.r 1.1 MB/s | 172 kB 00:00 2024-03-21T08:44:56.404 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:44:56.416 INFO:teuthology.orchestra.run.smithi105.stdout:(120/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 148 kB/s | 43 kB 00:00 2024-03-21T08:44:56.442 INFO:teuthology.orchestra.run.smithi105.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 3.5 MB/s | 358 kB 00:00 2024-03-21T08:44:56.448 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:44:56.467 INFO:teuthology.orchestra.run.smithi105.stdout:(122/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-03-21T08:44:56.467 INFO:teuthology.orchestra.run.smithi087.stdout:(101/174): qatlib-23.11.0-1.el9.x86_64.rpm 243 kB/s | 216 kB 00:00 2024-03-21T08:44:56.491 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:44:56.492 INFO:teuthology.orchestra.run.smithi105.stdout:(123/174): python3-colorama-0.4.4-12.el9.noarch 468 kB/s | 35 kB 00:00 2024-03-21T08:44:56.518 INFO:teuthology.orchestra.run.smithi105.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-03-21T08:44:56.520 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:44:56.576 INFO:teuthology.orchestra.run.smithi105.stdout:(125/174): python3-jaraco-8.2.1-3.el9.noarch.rp 184 kB/s | 11 kB 00:00 2024-03-21T08:44:56.588 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:44:56.601 INFO:teuthology.orchestra.run.smithi105.stdout:(126/174): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-03-21T08:44:56.634 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:44:56.652 INFO:teuthology.orchestra.run.smithi105.stdout:(127/174): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-03-21T08:44:56.677 INFO:teuthology.orchestra.run.smithi105.stdout:(128/174): python3-jaraco-classes-3.2.1-5.el9.n 176 kB/s | 18 kB 00:00 2024-03-21T08:44:56.702 INFO:teuthology.orchestra.run.smithi105.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-03-21T08:44:56.707 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:44:56.727 INFO:teuthology.orchestra.run.smithi105.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-03-21T08:44:56.761 INFO:teuthology.orchestra.run.smithi105.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 337 kB/s | 20 kB 00:00 2024-03-21T08:44:56.786 INFO:teuthology.orchestra.run.smithi105.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 792 kB/s | 46 kB 00:00 2024-03-21T08:44:56.820 INFO:teuthology.orchestra.run.smithi105.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 1.3 MB/s | 79 kB 00:00 2024-03-21T08:44:56.840 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:44:56.845 INFO:teuthology.orchestra.run.smithi105.stdout:(134/174): python3-natsort-7.1.1-5.el9.noarch.r 985 kB/s | 58 kB 00:00 2024-03-21T08:44:56.879 INFO:teuthology.orchestra.run.smithi105.stdout:(135/174): python3-kubernetes-26.1.0-2.el9.noar 5.8 MB/s | 1.0 MB 00:00 2024-03-21T08:44:56.894 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:44:56.912 INFO:teuthology.orchestra.run.smithi105.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 246 kB/s | 16 kB 00:00 2024-03-21T08:44:56.934 INFO:teuthology.orchestra.run.smithi087.stdout:(102/174): qatzip-libs-1.1.2-2.el9.x86_64.rpm 72 kB/s | 44 kB 00:00 2024-03-21T08:44:56.971 INFO:teuthology.orchestra.run.smithi105.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-03-21T08:44:56.996 INFO:teuthology.orchestra.run.smithi105.stdout:(138/174): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-03-21T08:44:57.022 INFO:teuthology.orchestra.run.smithi105.stdout:(139/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 630 kB/s | 90 kB 00:00 2024-03-21T08:44:57.047 INFO:teuthology.orchestra.run.smithi105.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-03-21T08:44:57.072 INFO:teuthology.orchestra.run.smithi105.stdout:(141/174): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-03-21T08:44:57.097 INFO:teuthology.orchestra.run.smithi105.stdout:(142/174): python3-rsa-4.9-2.el9.noarch.rpm 783 kB/s | 59 kB 00:00 2024-03-21T08:44:57.101 INFO:teuthology.orchestra.run.smithi087.stdout:(103/174): lua-devel-5.4.4-4.el9.x86_64.rpm 134 kB/s | 22 kB 00:00 2024-03-21T08:44:57.123 INFO:teuthology.orchestra.run.smithi105.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-03-21T08:44:57.148 INFO:teuthology.orchestra.run.smithi105.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-03-21T08:44:57.174 INFO:teuthology.orchestra.run.smithi105.stdout:(145/174): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-03-21T08:44:57.190 INFO:teuthology.orchestra.run.smithi105.stdout:(146/174): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-03-21T08:44:57.209 INFO:teuthology.orchestra.run.smithi087.stdout:(104/174): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 126 kB/s | 94 kB 00:00 2024-03-21T08:44:57.249 INFO:teuthology.orchestra.run.smithi105.stdout:(147/174): python3-werkzeug-2.0.3-3.el9.1.noarc 4.2 MB/s | 427 kB 00:00 2024-03-21T08:44:57.260 INFO:teuthology.orchestra.run.smithi087.stdout:(105/174): protobuf-compiler-3.14.0-13.el9.x86_ 5.3 MB/s | 863 kB 00:00 2024-03-21T08:44:57.265 INFO:teuthology.orchestra.run.smithi105.stdout:(148/174): python3-zc-lockfile-2.0-10.el9.noarc 217 kB/s | 20 kB 00:00 2024-03-21T08:44:57.291 INFO:teuthology.orchestra.run.smithi105.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 1.9 MB/s | 192 kB 00:00 2024-03-21T08:44:57.383 INFO:teuthology.orchestra.run.smithi105.stdout:(150/174): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-03-21T08:44:57.419 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:44:57.425 INFO:teuthology.orchestra.run.smithi105.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.7 MB/s | 321 kB 00:00 2024-03-21T08:44:57.499 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:44:57.567 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:44:57.609 INFO:teuthology.orchestra.run.smithi105.stdout:(152/174): librados2-19.0.0-2231.g2edfee68.el9. 9.9 MB/s | 3.4 MB 00:00 2024-03-21T08:44:57.619 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:44:57.627 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:44:57.651 INFO:teuthology.orchestra.run.smithi105.stdout:(153/174): NetworkManager-1.47.2-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-03-21T08:44:57.659 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:44:57.701 INFO:teuthology.orchestra.run.smithi105.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 844 kB/s | 42 kB 00:00 2024-03-21T08:44:57.708 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:44:57.746 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:44:57.751 INFO:teuthology.orchestra.run.smithi105.stdout:(155/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 6.8 MB/s | 3.1 MB 00:00 2024-03-21T08:44:57.775 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:44:57.777 INFO:teuthology.orchestra.run.smithi087.stdout:(106/174): gperftools-libs-2.9.1-2.el9.x86_64.r 598 kB/s | 309 kB 00:00 2024-03-21T08:44:57.802 INFO:teuthology.orchestra.run.smithi087.stdout:(107/174): abseil-cpp-20211102.0-3.el9.x86_64.r 926 kB/s | 548 kB 00:00 2024-03-21T08:44:57.834 INFO:teuthology.orchestra.run.smithi105.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 856 kB/s | 71 kB 00:00 2024-03-21T08:44:57.877 INFO:teuthology.orchestra.run.smithi105.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 873 kB/s | 36 kB 00:00 2024-03-21T08:44:57.943 INFO:teuthology.orchestra.run.smithi105.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 2.8 MB/s | 192 kB 00:00 2024-03-21T08:44:57.968 INFO:teuthology.orchestra.run.smithi105.stdout:(159/174): NetworkManager-tui-1.47.2-1.el9.x86_ 928 kB/s | 248 kB 00:00 2024-03-21T08:44:57.994 INFO:teuthology.orchestra.run.smithi105.stdout:(160/174): elfutils-libs-0.190-2.el9.x86_64.rpm 5.0 MB/s | 256 kB 00:00 2024-03-21T08:44:58.006 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:44:58.036 INFO:teuthology.orchestra.run.smithi105.stdout:(161/174): NetworkManager-libnm-1.47.2-1.el9.x8 4.3 MB/s | 1.8 MB 00:00 2024-03-21T08:44:58.044 INFO:teuthology.orchestra.run.smithi087.stdout:(108/174): grpc-data-1.46.7-10.el9.noarch.rpm 73 kB/s | 19 kB 00:00 2024-03-21T08:44:58.061 INFO:teuthology.orchestra.run.smithi105.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-21T08:44:58.086 INFO:teuthology.orchestra.run.smithi105.stdout:(163/174): libacl-2.3.1-4.el9.x86_64.rpm 196 kB/s | 23 kB 00:00 2024-03-21T08:44:58.120 INFO:teuthology.orchestra.run.smithi105.stdout:(164/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 155 kB 00:00 2024-03-21T08:44:58.145 INFO:teuthology.orchestra.run.smithi105.stdout:(165/174): libmount-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 136 kB 00:00 2024-03-21T08:44:58.180 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:44:58.273 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:44:58.303 INFO:teuthology.orchestra.run.smithi087.stdout:(109/174): libarrow-doc-9.0.0-10.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-03-21T08:44:58.315 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:44:58.461 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:44:58.462 INFO:teuthology.orchestra.run.smithi105.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 170 kB/s | 64 kB 00:00 2024-03-21T08:44:58.502 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:44:58.504 INFO:teuthology.orchestra.run.smithi105.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 75 kB/s | 29 kB 00:00 2024-03-21T08:44:58.557 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:44:58.653 INFO:teuthology.orchestra.run.smithi087.stdout:(110/174): liboath-2.6.7-2.el9.x86_64.rpm 139 kB/s | 49 kB 00:00 2024-03-21T08:44:58.720 INFO:teuthology.orchestra.run.smithi087.stdout:(111/174): libarrow-9.0.0-10.el9.x86_64.rpm 4.8 MB/s | 4.4 MB 00:00 2024-03-21T08:44:58.793 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:44:58.837 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:44:58.882 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:44:58.937 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:44:58.981 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:44:58.995 INFO:teuthology.orchestra.run.smithi087.stdout:(112/174): libunwind-1.6.2-1.el9.x86_64.rpm 197 kB/s | 67 kB 00:00 2024-03-21T08:44:59.024 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:44:59.066 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:44:59.145 INFO:teuthology.orchestra.run.smithi087.stdout:(113/174): luarocks-3.9.2-1.el9.noarch.rpm 355 kB/s | 151 kB 00:00 2024-03-21T08:44:59.198 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:44:59.247 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:44:59.277 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:44:59.318 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:44:59.411 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:44:59.452 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:44:59.509 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:44:59.543 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:44:59.568 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:44:59.596 INFO:teuthology.orchestra.run.smithi087.stdout:(114/174): parquet-libs-9.0.0-10.el9.x86_64.rpm 1.4 MB/s | 838 kB 00:00 2024-03-21T08:44:59.639 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:44:59.729 INFO:teuthology.orchestra.run.smithi087.stdout:(115/174): python3-asyncssh-2.13.2-5.el9.noarch 939 kB/s | 548 kB 00:00 2024-03-21T08:44:59.817 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:44:59.916 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:44:59.938 INFO:teuthology.orchestra.run.smithi087.stdout:(116/174): python3-bcrypt-3.2.2-1.el9.x86_64.rp 127 kB/s | 43 kB 00:00 2024-03-21T08:44:59.959 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:44:59.997 INFO:teuthology.orchestra.run.smithi087.stdout:(117/174): openssl-devel-3.0.7-27.el9.x86_64.rp 745 kB/s | 4.1 MB 00:05 2024-03-21T08:45:00.008 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:45:00.021 INFO:teuthology.orchestra.run.smithi087.stdout:(118/174): python3-cachetools-4.2.4-1.el9.noarc 110 kB/s | 32 kB 00:00 2024-03-21T08:45:00.071 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:45:00.106 INFO:teuthology.orchestra.run.smithi105.stdout:(168/174): openssl-3.0.7-27.el9.x86_64.rpm 632 kB/s | 1.2 MB 00:01 2024-03-21T08:45:00.148 INFO:teuthology.orchestra.run.smithi105.stdout:(169/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 1.3 MB/s | 2.2 MB 00:01 2024-03-21T08:45:00.173 INFO:teuthology.orchestra.run.smithi105.stdout:(170/174): util-linux-core-2.37.4-18.el9.x86_64 6.8 MB/s | 465 kB 00:00 2024-03-21T08:45:00.199 INFO:teuthology.orchestra.run.smithi105.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 620 kB/s | 31 kB 00:00 2024-03-21T08:45:00.214 INFO:teuthology.orchestra.run.smithi087.stdout:(119/174): python3-certifi-2021.10.8-2.el9.noar 54 kB/s | 15 kB 00:00 2024-03-21T08:45:00.262 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:45:00.305 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:45:00.307 INFO:teuthology.orchestra.run.smithi105.stdout:(172/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 687 kB/s | 74 kB 00:00 2024-03-21T08:45:00.422 INFO:teuthology.orchestra.run.smithi087.stdout:(120/174): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-03-21T08:45:00.432 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:45:00.470 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:45:00.510 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:45:00.550 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:45:00.579 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:45:00.606 INFO:teuthology.orchestra.run.smithi087.stdout:(121/174): python3-cherrypy-18.6.1-2.el9.noarch 614 kB/s | 358 kB 00:00 2024-03-21T08:45:00.629 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:45:00.668 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:45:00.689 INFO:teuthology.orchestra.run.smithi087.stdout:(122/174): python3-colorama-0.4.4-12.el9.noarch 132 kB/s | 35 kB 00:00 2024-03-21T08:45:00.699 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:45:00.715 INFO:teuthology.orchestra.run.smithi087.stdout:(123/174): python3-cmd2-2.4.2-2.el9.noarch.rpm 605 kB/s | 303 kB 00:00 2024-03-21T08:45:00.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:45:00.813 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:45:00.848 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:45:00.882 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:45:00.923 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:45:00.967 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:45:01.062 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:45:01.115 INFO:teuthology.orchestra.run.smithi087.stdout:(124/174): python3-google-auth-2.28.2-1.el9.noa 398 kB/s | 203 kB 00:00 2024-03-21T08:45:01.140 INFO:teuthology.orchestra.run.smithi087.stdout:(125/174): python3-grpcio-tools-1.46.7-10.el9.x 340 kB/s | 144 kB 00:00 2024-03-21T08:45:01.157 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:45:01.205 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:45:01.266 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:45:01.299 INFO:teuthology.orchestra.run.smithi087.stdout:(126/174): python3-jaraco-8.2.1-3.el9.noarch.rp 58 kB/s | 11 kB 00:00 2024-03-21T08:45:01.309 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:45:01.406 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:45:01.432 INFO:teuthology.orchestra.run.smithi087.stdout:(127/174): python3-jaraco-classes-3.2.1-5.el9.n 60 kB/s | 18 kB 00:00 2024-03-21T08:45:01.464 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:45:01.474 INFO:teuthology.orchestra.run.smithi087.stdout:(128/174): python3-grpcio-1.46.7-10.el9.x86_64. 2.6 MB/s | 2.0 MB 00:00 2024-03-21T08:45:01.566 INFO:teuthology.orchestra.run.smithi087.stdout:(129/174): python3-jaraco-collections-3.0.0-8.e 87 kB/s | 23 kB 00:00 2024-03-21T08:45:01.682 INFO:teuthology.orchestra.run.smithi105.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 728 kB/s | 1.1 MB 00:01 2024-03-21T08:45:01.691 INFO:teuthology.orchestra.run.smithi087.stdout:(130/174): python3-jaraco-functools-3.5.0-2.el9 75 kB/s | 19 kB 00:00 2024-03-21T08:45:01.742 INFO:teuthology.orchestra.run.smithi087.stdout:(131/174): python3-jaraco-text-3.2.0-6.el9.noar 74 kB/s | 20 kB 00:00 2024-03-21T08:45:02.042 INFO:teuthology.orchestra.run.smithi087.stdout:(132/174): python3-logutils-0.3.5-21.el9.noarch 132 kB/s | 46 kB 00:00 2024-03-21T08:45:02.084 INFO:teuthology.orchestra.run.smithi087.stdout:(133/174): python3-more-itertools-8.12.0-2.el9. 230 kB/s | 79 kB 00:00 2024-03-21T08:45:02.193 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:45:02.236 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:45:02.242 INFO:teuthology.orchestra.run.smithi087.stdout:(134/174): python3-kubernetes-26.1.0-2.el9.noar 1.5 MB/s | 1.0 MB 00:00 2024-03-21T08:45:02.269 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:45:02.317 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:45:02.376 INFO:teuthology.orchestra.run.smithi087.stdout:(135/174): python3-natsort-7.1.1-5.el9.noarch.r 173 kB/s | 58 kB 00:00 2024-03-21T08:45:02.396 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:45:02.437 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:45:02.501 INFO:teuthology.orchestra.run.smithi087.stdout:(136/174): python3-portend-3.1.0-2.el9.noarch.r 64 kB/s | 16 kB 00:00 2024-03-21T08:45:02.613 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:45:02.652 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:45:02.668 INFO:teuthology.orchestra.run.smithi087.stdout:(137/174): python3-pecan-1.4.2-3.el9.noarch.rpm 466 kB/s | 272 kB 00:00 2024-03-21T08:45:02.711 INFO:teuthology.orchestra.run.smithi087.stdout:(138/174): python3-pyOpenSSL-21.0.0-1.el9.noarc 269 kB/s | 90 kB 00:00 2024-03-21T08:45:02.729 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:45:02.752 INFO:teuthology.orchestra.run.smithi087.stdout:(139/174): python3-pyperclip-1.8.2-2.el9.noarch 113 kB/s | 28 kB 00:00 2024-03-21T08:45:02.928 INFO:teuthology.orchestra.run.smithi087.stdout:(140/174): python3-repoze-lru-0.7-16.el9.noarch 119 kB/s | 31 kB 00:00 2024-03-21T08:45:03.095 INFO:teuthology.orchestra.run.smithi087.stdout:(141/174): python3-rsa-4.9-2.el9.noarch.rpm 173 kB/s | 59 kB 00:00 2024-03-21T08:45:03.145 INFO:teuthology.orchestra.run.smithi087.stdout:(142/174): python3-routes-2.5.1-5.el9.noarch.rp 434 kB/s | 188 kB 00:00 2024-03-21T08:45:03.195 INFO:teuthology.orchestra.run.smithi087.stdout:(143/174): python3-tempora-5.0.0-2.el9.noarch.r 134 kB/s | 36 kB 00:00 2024-03-21T08:45:03.400 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:45:03.462 INFO:teuthology.orchestra.run.smithi087.stdout:(144/174): python3-typing-extensions-4.4.0-2.el 139 kB/s | 51 kB 00:00 2024-03-21T08:45:03.506 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:45:03.537 INFO:teuthology.orchestra.run.smithi087.stdout:(145/174): python3-websocket-client-1.2.3-2.el9 262 kB/s | 90 kB 00:00 2024-03-21T08:45:03.654 INFO:teuthology.orchestra.run.smithi087.stdout:(146/174): python3-webob-1.8.7-6.el9.noarch.rpm 451 kB/s | 230 kB 00:00 2024-03-21T08:45:03.804 INFO:teuthology.orchestra.run.smithi087.stdout:(147/174): python3-zc-lockfile-2.0-10.el9.noarc 75 kB/s | 20 kB 00:00 2024-03-21T08:45:03.957 INFO:teuthology.orchestra.run.smithi105.stdout:(174/174): util-linux-2.37.4-18.el9.x86_64.rpm 429 kB/s | 2.3 MB 00:05 2024-03-21T08:45:03.964 INFO:teuthology.orchestra.run.smithi105.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:45:03.964 INFO:teuthology.orchestra.run.smithi105.stdout:Total 12 MB/s | 224 MB 00:18 2024-03-21T08:45:04.054 INFO:teuthology.orchestra.run.smithi087.stdout:(148/174): python3-werkzeug-2.0.3-3.el9.1.noarc 721 kB/s | 427 kB 00:00 2024-03-21T08:45:04.080 INFO:teuthology.orchestra.run.smithi087.stdout:(149/174): re2-20211101-3.el9.x86_64.rpm 450 kB/s | 192 kB 00:00 2024-03-21T08:45:04.436 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction check 2024-03-21T08:45:04.455 INFO:teuthology.orchestra.run.smithi087.stdout:(150/174): librbd1-19.0.0-2231.g2edfee68.el9.x8 8.4 MB/s | 3.1 MB 00:00 2024-03-21T08:45:04.497 INFO:teuthology.orchestra.run.smithi087.stdout:(151/174): xfsdump-3.1.12-4.el9.x86_64.rpm 7.6 MB/s | 321 kB 00:00 2024-03-21T08:45:04.572 INFO:teuthology.orchestra.run.smithi087.stdout:(152/174): thrift-0.15.0-2.el9.x86_64.rpm 2.1 MB/s | 1.6 MB 00:00 2024-03-21T08:45:04.706 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction check succeeded. 2024-03-21T08:45:04.706 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction test 2024-03-21T08:45:04.718 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:45:04.794 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:45:04.826 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:45:04.840 INFO:teuthology.orchestra.run.smithi087.stdout:(153/174): librados2-19.0.0-2231.g2edfee68.el9. 4.3 MB/s | 3.4 MB 00:00 2024-03-21T08:45:04.857 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:45:04.884 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:45:05.006 INFO:teuthology.orchestra.run.smithi087.stdout:(154/174): NetworkManager-team-1.47.2-1.el9.x86 253 kB/s | 42 kB 00:00 2024-03-21T08:45:05.131 INFO:teuthology.orchestra.run.smithi087.stdout:(155/174): NetworkManager-tui-1.47.2-1.el9.x86_ 1.9 MB/s | 248 kB 00:00 2024-03-21T08:45:05.190 INFO:teuthology.orchestra.run.smithi087.stdout:(156/174): acl-2.3.1-4.el9.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2024-03-21T08:45:05.194 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:45:05.231 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:45:05.249 INFO:teuthology.orchestra.run.smithi087.stdout:(157/174): elfutils-debuginfod-client-0.190-2.e 627 kB/s | 36 kB 00:00 2024-03-21T08:45:05.305 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:45:05.307 INFO:teuthology.orchestra.run.smithi087.stdout:(158/174): elfutils-libelf-0.190-2.el9.x86_64.r 3.2 MB/s | 192 kB 00:00 2024-03-21T08:45:05.336 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:45:05.372 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:45:05.374 INFO:teuthology.orchestra.run.smithi087.stdout:(159/174): elfutils-libs-0.190-2.el9.x86_64.rpm 3.8 MB/s | 256 kB 00:00 2024-03-21T08:45:05.433 INFO:teuthology.orchestra.run.smithi087.stdout:(160/174): libacl-2.3.1-4.el9.x86_64.rpm 394 kB/s | 23 kB 00:00 2024-03-21T08:45:05.474 INFO:teuthology.orchestra.run.smithi087.stdout:(161/174): NetworkManager-libnm-1.47.2-1.el9.x8 2.0 MB/s | 1.8 MB 00:00 2024-03-21T08:45:05.500 INFO:teuthology.orchestra.run.smithi087.stdout:(162/174): libblkid-2.37.4-18.el9.x86_64.rpm 1.6 MB/s | 108 kB 00:00 2024-03-21T08:45:05.550 INFO:teuthology.orchestra.run.smithi087.stdout:(163/174): NetworkManager-1.47.2-1.el9.x86_64.r 2.2 MB/s | 2.3 MB 00:01 2024-03-21T08:45:05.576 INFO:teuthology.orchestra.run.smithi087.stdout:(164/174): libmount-2.37.4-18.el9.x86_64.rpm 1.8 MB/s | 136 kB 00:00 2024-03-21T08:45:05.592 INFO:teuthology.orchestra.run.smithi087.stdout:(165/174): libfdisk-2.37.4-18.el9.x86_64.rpm 1.3 MB/s | 155 kB 00:00 2024-03-21T08:45:05.618 INFO:teuthology.orchestra.run.smithi087.stdout:(166/174): libsmartcols-2.37.4-18.el9.x86_64.rp 954 kB/s | 64 kB 00:00 2024-03-21T08:45:05.643 INFO:teuthology.orchestra.run.smithi087.stdout:(167/174): libuuid-2.37.4-18.el9.x86_64.rpm 426 kB/s | 29 kB 00:00 2024-03-21T08:45:05.826 INFO:teuthology.orchestra.run.smithi087.stdout:(168/174): util-linux-2.37.4-18.el9.x86_64.rpm 12 MB/s | 2.3 MB 00:00 2024-03-21T08:45:05.832 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:45:05.902 INFO:teuthology.orchestra.run.smithi087.stdout:(169/174): util-linux-core-2.37.4-18.el9.x86_64 6.0 MB/s | 465 kB 00:00 2024-03-21T08:45:05.935 INFO:teuthology.orchestra.run.smithi087.stdout:(170/174): openssl-3.0.7-27.el9.x86_64.rpm 3.5 MB/s | 1.2 MB 00:00 2024-03-21T08:45:05.961 INFO:teuthology.orchestra.run.smithi087.stdout:(171/174): util-linux-user-2.37.4-18.el9.x86_64 534 kB/s | 31 kB 00:00 2024-03-21T08:45:06.036 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:45:06.136 INFO:teuthology.orchestra.run.smithi087.stdout:(172/174): openssl-libs-3.0.7-27.el9.x86_64.rpm 4.2 MB/s | 2.2 MB 00:00 2024-03-21T08:45:06.203 INFO:teuthology.orchestra.run.smithi087.stdout:(173/174): xfsprogs-6.3.0-1.el9.x86_64.rpm 4.0 MB/s | 1.1 MB 00:00 2024-03-21T08:45:06.238 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:45:06.261 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:45:06.439 INFO:teuthology.orchestra.run.smithi105.stdout:Transaction test succeeded. 2024-03-21T08:45:06.440 INFO:teuthology.orchestra.run.smithi105.stdout:Running transaction 2024-03-21T08:45:06.703 INFO:teuthology.orchestra.run.smithi087.stdout:(174/174): libacl-devel-2.3.1-4.el9.x86_64.rpm 100 kB/s | 74 kB 00:00 2024-03-21T08:45:06.708 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:45:06.709 INFO:teuthology.orchestra.run.smithi087.stdout:Total 7.8 MB/s | 224 MB 00:28 2024-03-21T08:45:07.192 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction check 2024-03-21T08:45:07.467 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction check succeeded. 2024-03-21T08:45:07.467 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction test 2024-03-21T08:45:08.484 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:08.524 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:09.059 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:09.103 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:45:09.175 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:45:09.176 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:45:09.176 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:45:09.176 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:09.204 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:45:09.494 INFO:teuthology.orchestra.run.smithi087.stdout:Transaction test succeeded. 2024-03-21T08:45:09.495 INFO:teuthology.orchestra.run.smithi087.stdout:Running transaction 2024-03-21T08:45:13.368 INFO:teuthology.orchestra.run.smithi105.stdout: Preparing : 1/1 2024-03-21T08:45:13.469 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:45:13.730 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:45:13.776 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:45:13.787 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:45:13.827 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:45:13.889 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:45:13.930 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:45:13.963 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:45:14.007 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:45:14.335 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:45:14.390 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:45:14.519 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:45:14.557 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:45:14.648 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:45:14.678 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:45:14.715 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:45:14.885 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:45:14.935 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:45:14.971 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:45:15.065 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:45:15.088 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:45:15.152 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:45:15.192 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:45:15.215 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:45:15.231 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:45:15.415 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:45:15.446 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:45:15.469 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:45:15.571 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:45:15.672 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:45:15.732 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:45:15.808 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:45:15.858 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:45:16.073 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:45:16.205 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:45:16.298 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:45:16.330 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:45:16.356 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:45:16.391 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:45:16.409 INFO:teuthology.orchestra.run.smithi087.stdout: Preparing : 1/1 2024-03-21T08:45:16.465 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:45:16.499 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-libs-1:3.0.7-27.el9.x86_64 1/198 2024-03-21T08:45:16.519 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:45:16.542 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libuuid-2.37.4-18.el9.x86_64 2/198 2024-03-21T08:45:16.566 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:45:16.580 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:45:16.591 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libblkid-2.37.4-18.el9.x86_64 3/198 2024-03-21T08:45:16.629 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:45:16.635 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 4/198 2024-03-21T08:45:16.665 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:45:16.692 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 5/198 2024-03-21T08:45:16.709 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:45:16.725 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/198 2024-03-21T08:45:16.741 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:45:16.751 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 7/198 2024-03-21T08:45:16.769 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:45:16.802 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 8/198 2024-03-21T08:45:16.838 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:45:16.875 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:45:16.917 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:45:16.947 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:45:17.015 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:45:17.069 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:45:17.130 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 9/198 2024-03-21T08:45:17.142 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:45:17.200 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:45:17.275 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:45:17.329 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:45:17.330 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:45:17.351 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:45:17.445 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:45:17.476 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:45:17.524 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 13/198 2024-03-21T08:45:17.693 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-libelf-0.190-2.el9.x86_64 14/198 2024-03-21T08:45:17.744 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-13.11-1.el9.x86_64 15/198 2024-03-21T08:45:17.773 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:45:17.842 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:45:17.876 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:45:17.899 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:45:17.925 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:45:17.964 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 17/198 2024-03-21T08:45:17.999 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:45:18.003 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libmount-2.37.4-18.el9.x86_64 18/198 2024-03-21T08:45:18.025 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:45:18.037 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:45:18.041 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-service-23.11.0-1.el9.x86_64 19/198 2024-03-21T08:45:18.045 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:45:18.077 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:45:18.118 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:45:18.156 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:45:18.185 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:45:18.226 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:45:18.257 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatlib-23.11.0-1.el9.x86_64 20/198 2024-03-21T08:45:18.305 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libsmartcols-2.37.4-18.el9.x86_64 21/198 2024-03-21T08:45:18.415 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:45:18.490 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 22/198 2024-03-21T08:45:18.571 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:45:18.611 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:45:18.629 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:45:18.654 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 23/198 2024-03-21T08:45:18.673 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:45:18.748 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:45:18.796 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:45:18.804 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:45:18.860 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:45:18.914 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:45:18.978 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:45:19.010 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:45:19.016 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.47.2-1.el9.x86_64 24/198 2024-03-21T08:45:19.047 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:45:19.094 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:45:19.117 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : re2-1:20211101-3.el9.x86_64 25/198 2024-03-21T08:45:19.130 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:45:19.175 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:45:19.205 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 26/198 2024-03-21T08:45:19.216 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:45:19.243 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 27/198 2024-03-21T08:45:19.270 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : liboath-2.6.7-2.el9.x86_64 28/198 2024-03-21T08:45:19.305 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/198 2024-03-21T08:45:19.378 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:45:19.379 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/198 2024-03-21T08:45:19.422 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:45:19.445 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 31/198 2024-03-21T08:45:19.452 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:45:19.480 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 32/198 2024-03-21T08:45:19.493 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:45:19.553 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 33/198 2024-03-21T08:45:19.588 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 34/198 2024-03-21T08:45:19.588 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:45:19.630 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 35/198 2024-03-21T08:45:19.652 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:45:19.664 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 36/198 2024-03-21T08:45:19.691 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-5.4.4-4.el9.x86_64 37/198 2024-03-21T08:45:19.691 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:45:19.718 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:45:19.743 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:45:19.760 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 38/198 2024-03-21T08:45:19.804 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 39/198 2024-03-21T08:45:19.813 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:45:19.847 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 40/198 2024-03-21T08:45:19.876 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 41/198 2024-03-21T08:45:19.945 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/198 2024-03-21T08:45:19.949 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:45:19.991 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : luarocks-3.9.2-1.el9.noarch 43/198 2024-03-21T08:45:20.060 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:45:20.073 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 44/198 2024-03-21T08:45:20.111 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:45:20.158 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:45:20.206 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 45/198 2024-03-21T08:45:20.213 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:45:20.259 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 46/198 2024-03-21T08:45:20.398 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:45:20.455 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:45:20.583 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:45:20.619 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:45:20.651 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:45:20.683 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:45:20.711 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:45:20.745 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:45:20.774 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rsa-4.9-2.el9.noarch 47/198 2024-03-21T08:45:20.784 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:45:20.816 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:45:20.854 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 48/198 2024-03-21T08:45:20.862 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:45:20.900 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:45:20.900 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /sys 2024-03-21T08:45:20.900 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /proc 2024-03-21T08:45:20.901 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /mnt 2024-03-21T08:45:20.901 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /var/tmp 2024-03-21T08:45:20.901 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /home 2024-03-21T08:45:20.901 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /root 2024-03-21T08:45:20.901 INFO:teuthology.orchestra.run.smithi100.stdout:skipping the directory /tmp 2024-03-21T08:45:20.901 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:20.938 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:45:20.956 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 49/198 2024-03-21T08:45:20.981 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:45:20.999 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:45:21.008 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: util-linux-core-2.37.4-18.el9.x86_64 50/198 2024-03-21T08:45:21.014 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:45:21.056 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:45:21.059 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : qatzip-libs-1.1.2-2.el9.x86_64 51/198 2024-03-21T08:45:21.076 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:45:21.090 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:45:21.091 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:45:21.091 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:21.091 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:45:21.092 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:21.097 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-libs-0.190-2.el9.x86_64 52/198 2024-03-21T08:45:21.143 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 53/198 2024-03-21T08:45:21.173 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/198 2024-03-21T08:45:21.179 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:45:21.304 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:45:21.336 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:45:21.383 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:45:21.443 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libfdisk-2.37.4-18.el9.x86_64 55/198 2024-03-21T08:45:21.480 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:45:21.529 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:45:21.615 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : util-linux-2.37.4-18.el9.x86_64 56/198 2024-03-21T08:45:21.683 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : xfsprogs-6.3.0-1.el9.x86_64 57/198 2024-03-21T08:45:21.727 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 58/198 2024-03-21T08:45:21.868 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 59/198 2024-03-21T08:45:21.921 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : openssl-1:3.0.7-27.el9.x86_64 60/198 2024-03-21T08:45:21.976 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 61/198 2024-03-21T08:45:22.048 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 62/198 2024-03-21T08:45:22.080 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 63/198 2024-03-21T08:45:22.108 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 64/198 2024-03-21T08:45:22.155 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 65/198 2024-03-21T08:45:22.191 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 66/198 2024-03-21T08:45:22.234 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 67/198 2024-03-21T08:45:22.259 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:45:22.276 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 68/198 2024-03-21T08:45:22.309 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:45:22.343 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:45:22.391 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:45:22.409 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 69/198 2024-03-21T08:45:22.457 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 70/198 2024-03-21T08:45:22.478 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:45:22.486 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 71/198 2024-03-21T08:45:22.511 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:45:22.520 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 72/198 2024-03-21T08:45:22.571 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:45:22.597 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:45:22.597 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:22.597 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:45:22.597 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:45:22.597 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:45:22.597 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:22.613 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 73/198 2024-03-21T08:45:22.653 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 74/198 2024-03-21T08:45:22.685 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:45:22.694 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 75/198 2024-03-21T08:45:22.717 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:45:22.719 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 76/198 2024-03-21T08:45:22.744 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 77/198 2024-03-21T08:45:22.794 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:45:22.813 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:45:22.821 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 78/198 2024-03-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:22.943 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 79/198 2024-03-21T08:45:23.042 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 80/198 2024-03-21T08:45:23.085 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 81/198 2024-03-21T08:45:23.132 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-ply-3.11-14.el9.noarch 82/198 2024-03-21T08:45:23.189 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 83/198 2024-03-21T08:45:23.280 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:45:23.303 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:45:23.303 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:23.303 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:45:23.303 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:45:23.303 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:45:23.303 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:23.372 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 84/198 2024-03-21T08:45:23.399 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:45:23.422 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 85/198 2024-03-21T08:45:23.435 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:45:23.547 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:45:23.559 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/198 2024-03-21T08:45:23.595 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 87/198 2024-03-21T08:45:23.627 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 88/198 2024-03-21T08:45:23.658 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : fuse-2.9.9-15.el9.x86_64 89/198 2024-03-21T08:45:23.686 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 90/198 2024-03-21T08:45:23.720 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 91/198 2024-03-21T08:45:23.759 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : uuid-1.6.2-55.el9.x86_64 92/198 2024-03-21T08:45:23.790 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 93/198 2024-03-21T08:45:23.820 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 94/198 2024-03-21T08:45:23.849 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:45:23.880 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:45:23.896 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 95/198 2024-03-21T08:45:23.912 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:45:23.930 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 96/198 2024-03-21T08:45:23.964 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 97/198 2024-03-21T08:45:23.972 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:45:23.995 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:45:24.005 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 98/198 2024-03-21T08:45:24.040 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 99/198 2024-03-21T08:45:24.128 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 100/198 2024-03-21T08:45:24.218 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 101/198 2024-03-21T08:45:24.227 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:45:24.263 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:45:24.304 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 102/198 2024-03-21T08:45:24.336 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 103/198 2024-03-21T08:45:24.382 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 104/198 2024-03-21T08:45:24.479 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-packaging-20.9-5.el9.noarch 105/198 2024-03-21T08:45:24.529 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 106/198 2024-03-21T08:45:24.755 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:45:24.851 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:45:24.882 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:45:24.905 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:45:24.933 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:45:25.221 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:45:25.242 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 107/198 2024-03-21T08:45:25.244 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:45:25.257 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:45:25.271 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:45:25.283 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 108/198 2024-03-21T08:45:25.317 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 109/198 2024-03-21T08:45:25.359 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:45:25.360 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:45:25.365 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 110/198 2024-03-21T08:45:25.392 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:45:25.428 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:45:25.445 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 111/198 2024-03-21T08:45:25.476 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 112/198 2024-03-21T08:45:25.519 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:45:25.551 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:45:25.573 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:45:25.574 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:25.574 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:45:25.574 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:45:25.574 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:45:25.574 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:25.618 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:45:25.629 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:45:25.653 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-0.3.21-2.el9.x86_64 113/198 2024-03-21T08:45:25.677 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:45:25.682 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 114/198 2024-03-21T08:45:25.752 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 115/198 2024-03-21T08:45:25.887 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:45:25.915 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:45:25.986 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:45:26.009 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:45:26.396 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 116/198 2024-03-21T08:45:26.536 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 117/198 2024-03-21T08:45:26.541 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:45:26.599 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:45:26.621 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:45:26.622 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:26.622 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:45:26.622 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:45:26.622 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:45:26.622 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:26.663 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:45:26.684 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:45:26.685 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:26.685 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:45:26.685 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:26.738 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:26.761 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:26.761 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:26.761 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:45:26.761 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:45:26.761 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:45:26.761 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:27.081 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:45:27.104 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:45:27.104 INFO:teuthology.orchestra.run.smithi100.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:27.104 INFO:teuthology.orchestra.run.smithi100.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:45:27.104 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:45:27.104 INFO:teuthology.orchestra.run.smithi100.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:45:27.104 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:27.740 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 118/198 2024-03-21T08:45:27.821 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 119/198 2024-03-21T08:45:27.852 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 120/198 2024-03-21T08:45:27.883 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : jq-1.6-16.el9.x86_64 121/198 2024-03-21T08:45:27.911 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 122/198 2024-03-21T08:45:28.198 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:28.223 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 123/198 2024-03-21T08:45:28.239 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:28.249 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:45:28.339 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 124/198 2024-03-21T08:45:28.370 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 125/198 2024-03-21T08:45:28.407 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 126/198 2024-03-21T08:45:28.791 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:28.863 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 127/198 2024-03-21T08:45:28.893 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:45:28.909 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:45:28.948 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 128/198 2024-03-21T08:45:28.971 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 129/198 2024-03-21T08:45:28.990 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:45:28.990 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:45:28.990 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:45:28.990 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:29.018 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:45:31.189 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:31.225 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:31.536 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:45:31.585 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:45:31.615 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:45:31.668 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:45:31.768 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 130/198 2024-03-21T08:45:31.803 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:45:31.848 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:45:31.888 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 131/198 2024-03-21T08:45:31.888 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-03-21T08:45:31.889 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-03-21T08:45:31.889 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:31.929 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:45:31.967 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:45:32.007 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:45:32.046 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:45:32.107 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:45:32.151 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:45:32.183 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:45:32.224 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:45:32.253 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:45:32.310 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:45:32.373 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:45:32.411 INFO:teuthology.orchestra.run.smithi100.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:45:32.963 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:33.001 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:33.020 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:33.826 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:45:33.904 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:45:33.944 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:45:34.025 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:45:34.066 INFO:teuthology.orchestra.run.smithi100.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:45:34.091 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:45:34.146 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:45:34.176 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:45:34.176 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:45:34.266 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:45:34.302 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:45:34.302 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:45:34.395 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:45:34.422 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:45:34.455 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:45:34.480 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:45:34.505 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:45:34.538 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:45:34.578 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:45:34.618 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:45:34.644 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:45:34.669 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:45:34.695 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:45:34.720 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:45:34.726 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:34.727 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:34.923 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:34.923 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:45:35.014 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:45:35.039 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:45:35.070 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:45:35.098 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:45:35.123 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:45:35.187 INFO:teuthology.orchestra.run.smithi100.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:45:40.240 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:45:40.241 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:45:40.242 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:45:40.259 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:45:40.260 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:45:40.261 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:45:40.262 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:45:40.263 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:45:40.264 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:45:40.265 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:45:40.266 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:45:40.267 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:45:40.268 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:45:40.269 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:45:40.270 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:45:40.271 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /sys 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /proc 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /mnt 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /var/tmp 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /home 2024-03-21T08:45:40.873 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /root 2024-03-21T08:45:40.874 INFO:teuthology.orchestra.run.smithi105.stdout:skipping the directory /tmp 2024-03-21T08:45:40.874 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:41.040 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:45:41.055 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:45:41.055 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:41.055 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:45:41.056 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:42.535 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:45:42.561 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:45:42.561 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:42.561 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:45:42.561 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:45:42.561 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:45:42.561 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:42.777 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:45:42.799 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:45:42.799 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:42.799 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:45:42.799 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:45:42.799 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:45:42.799 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout:Upgraded: 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:45:43.122 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:45:43.123 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout:Installed: 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.124 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.125 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:45:43.126 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.127 INFO:teuthology.orchestra.run.smithi100.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:45:43.128 INFO:teuthology.orchestra.run.smithi100.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:45:43.129 INFO:teuthology.orchestra.run.smithi100.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:45:43.130 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:45:43.131 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:45:43.131 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:45:43.131 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:45:43.131 INFO:teuthology.orchestra.run.smithi100.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:45:43.132 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:45:43.133 INFO:teuthology.orchestra.run.smithi100.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:45:43.134 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:45:43.135 INFO:teuthology.orchestra.run.smithi100.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.136 INFO:teuthology.orchestra.run.smithi100.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:45:43.137 INFO:teuthology.orchestra.run.smithi100.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:45:43.137 INFO:teuthology.orchestra.run.smithi100.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:45:43.137 INFO:teuthology.orchestra.run.smithi100.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:45:43.137 INFO:teuthology.orchestra.run.smithi100.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:45:43.137 INFO:teuthology.orchestra.run.smithi100.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:45:43.137 INFO:teuthology.orchestra.run.smithi100.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:45:43.138 INFO:teuthology.orchestra.run.smithi100.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:45:43.138 INFO:teuthology.orchestra.run.smithi100.stdout: 2024-03-21T08:45:43.138 INFO:teuthology.orchestra.run.smithi100.stdout:Complete! 2024-03-21T08:45:43.236 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:45:43.258 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:45:43.259 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:43.259 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:45:43.259 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:45:43.259 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:45:43.259 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:43.354 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:45:43.467 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 132/198 2024-03-21T08:45:43.467 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /sys 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /proc 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /mnt 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /var/tmp 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /home 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /root 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout:skipping the directory /tmp 2024-03-21T08:45:43.468 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:43.612 DEBUG:teuthology.parallel:result is None 2024-03-21T08:45:43.633 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:45:43.647 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 133/198 2024-03-21T08:45:43.647 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:43.648 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-03-21T08:45:43.648 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:43.780 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:45:43.813 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:45:43.845 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:45:43.906 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:45:43.928 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:45:44.161 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:45:44.196 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:45:45.110 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:45:45.135 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 134/198 2024-03-21T08:45:45.135 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:45.136 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-03-21T08:45:45.136 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:45:45.136 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-03-21T08:45:45.136 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:45.158 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:45:45.179 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:45:45.278 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:45:45.353 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:45:45.375 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 135/198 2024-03-21T08:45:45.375 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:45.376 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-03-21T08:45:45.376 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:45:45.376 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-03-21T08:45:45.376 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:45.442 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:45:45.473 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:45:45.496 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:45:45.496 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:45.496 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:45:45.496 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:45:45.496 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:45:45.496 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:45.540 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:45:45.551 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:45:45.596 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:45:45.811 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:45:45.834 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 136/198 2024-03-21T08:45:45.834 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:45.834 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-03-21T08:45:45.834 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:45:45.834 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-03-21T08:45:45.834 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:45.929 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : automake-1.16.2-8.el9.noarch 137/198 2024-03-21T08:45:46.380 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:45:46.411 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 138/198 2024-03-21T08:45:46.443 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 139/198 2024-03-21T08:45:46.470 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:45:46.501 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:45:46.504 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 140/198 2024-03-21T08:45:46.523 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:45:46.524 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:46.524 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:45:46.524 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:45:46.524 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:45:46.524 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:46.534 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:45:46.569 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:45:46.590 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:45:46.590 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:46.590 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:45:46.590 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:46.635 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:46.658 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:46.658 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:46.658 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:45:46.658 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:45:46.658 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:45:46.658 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:46.766 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 141/198 2024-03-21T08:45:46.801 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:45:46.978 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:45:47.000 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:45:47.000 INFO:teuthology.orchestra.run.smithi105.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:47.000 INFO:teuthology.orchestra.run.smithi105.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:45:47.000 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:45:47.001 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:45:47.001 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:45:47.755 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 142/198 2024-03-21T08:45:47.785 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:45:47.885 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 143/198 2024-03-21T08:45:48.038 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 144/198 2024-03-21T08:45:48.071 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:45:48.094 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 145/198 2024-03-21T08:45:48.094 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:48.094 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-03-21T08:45:48.094 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:45:48.094 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-03-21T08:45:48.094 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:48.138 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:45:48.149 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 146/198 2024-03-21T08:45:48.196 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 147/198 2024-03-21T08:45:49.064 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libtool-2.4.6-45.el9.x86_64 148/198 2024-03-21T08:45:49.090 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:45:49.113 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 149/198 2024-03-21T08:45:49.113 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:49.113 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-03-21T08:45:49.113 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:45:49.113 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-03-21T08:45:49.113 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:49.157 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:45:49.179 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 150/198 2024-03-21T08:45:49.179 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:49.179 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-03-21T08:45:49.179 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:49.233 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:49.255 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-03-21T08:45:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:45:49.255 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-03-21T08:45:49.255 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:49.566 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:45:49.589 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 152/198 2024-03-21T08:45:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-03-21T08:45:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-03-21T08:45:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:45:49.589 INFO:teuthology.orchestra.run.smithi087.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-03-21T08:45:49.589 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:45:51.434 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:45:51.474 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:45:51.504 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:45:51.531 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:45:51.712 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:45:51.831 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:45:51.871 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:45:51.910 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:45:51.963 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:45:52.015 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:45:52.047 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:45:52.088 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:45:52.117 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:45:52.180 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:45:52.241 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:45:52.273 INFO:teuthology.orchestra.run.smithi105.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:45:52.811 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:52.855 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:52.875 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:53.696 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:45:53.775 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:45:53.815 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi105.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:45:53.954 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:45:53.980 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:45:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:45:54.006 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:45:54.078 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 154/198 2024-03-21T08:45:54.093 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:45:54.112 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 155/198 2024-03-21T08:45:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:45:54.123 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:45:54.145 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 156/198 2024-03-21T08:45:54.208 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:45:54.234 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:45:54.268 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:45:54.293 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:45:54.317 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:45:54.348 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 157/198 2024-03-21T08:45:54.350 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:45:54.390 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:45:54.423 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:45:54.448 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:45:54.468 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 158/198 2024-03-21T08:45:54.473 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:45:54.498 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:45:54.508 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 159/198 2024-03-21T08:45:54.524 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:45:54.530 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:54.531 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:54.566 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 160/198 2024-03-21T08:45:54.659 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 161/198 2024-03-21T08:45:54.703 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : util-linux-user-2.37.4-18.el9.x86_64 162/198 2024-03-21T08:45:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:54.718 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:45:54.735 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 163/198 2024-03-21T08:45:54.770 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : elfutils-debuginfod-client-0.190-2.el9.x86_64 164/198 2024-03-21T08:45:54.797 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-team-1:1.47.2-1.el9.x86_64 165/198 2024-03-21T08:45:54.801 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:45:54.826 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:45:54.858 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:45:54.880 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : NetworkManager-tui-1:1.47.2-1.el9.x86_64 166/198 2024-03-21T08:45:54.885 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:45:54.910 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:45:54.940 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 167/198 2024-03-21T08:45:54.974 INFO:teuthology.orchestra.run.smithi105.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:45:54.980 INFO:teuthology.orchestra.run.smithi087.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 168/198 2024-03-21T08:45:55.572 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:55.634 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:55.653 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 169/198 2024-03-21T08:45:56.467 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 170/198 2024-03-21T08:45:56.556 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 171/198 2024-03-21T08:45:56.596 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 172/198 2024-03-21T08:45:56.679 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : flex-2.6.4-9.el9.x86_64 173/198 2024-03-21T08:45:56.709 INFO:teuthology.orchestra.run.smithi087.stdout: Installing : bison-3.7.4-5.el9.x86_64 174/198 2024-03-21T08:45:56.734 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 175/198 2024-03-21T08:45:56.759 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 176/198 2024-03-21T08:45:56.786 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 177/198 2024-03-21T08:45:56.786 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:45:56.882 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 178/198 2024-03-21T08:45:56.912 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 179/198 2024-03-21T08:45:56.912 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:45:56.988 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 180/198 2024-03-21T08:45:57.016 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 181/198 2024-03-21T08:45:57.048 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : xfsprogs-5.19.0-4.el9.x86_64 182/198 2024-03-21T08:45:57.073 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 183/198 2024-03-21T08:45:57.098 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 184/198 2024-03-21T08:45:57.144 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : util-linux-user-2.37.4-15.el9.x86_64 185/198 2024-03-21T08:45:57.187 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : util-linux-2.37.4-15.el9.x86_64 186/198 2024-03-21T08:45:57.222 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : util-linux-core-2.37.4-15.el9.x86_64 187/198 2024-03-21T08:45:57.261 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libfdisk-2.37.4-15.el9.x86_64 188/198 2024-03-21T08:45:57.286 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libmount-2.37.4-15.el9.x86_64 189/198 2024-03-21T08:45:57.320 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libblkid-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:45:57.345 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 191/198 2024-03-21T08:45:57.351 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:57.352 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:57.531 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 192/198 2024-03-21T08:45:57.531 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:45:57.622 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 193/198 2024-03-21T08:45:57.647 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libuuid-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:45:57.679 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libsmartcols-2.37.4-15.el9.x86_64 195/198 2024-03-21T08:45:57.707 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 196/198 2024-03-21T08:45:57.732 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 197/198 2024-03-21T08:45:57.797 INFO:teuthology.orchestra.run.smithi087.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:45:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:45:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:45:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:45:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:45:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:45:59.694 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:45:59.695 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:45:59.696 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:45:59.712 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:45:59.713 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:45:59.714 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:45:59.715 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:45:59.716 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:45:59.717 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:45:59.718 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:45:59.719 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:45:59.720 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:45:59.721 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:45:59.722 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:45:59.723 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:45:59.724 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:46:02.552 INFO:teuthology.orchestra.run.smithi105.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:46:02.552 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:46:02.552 INFO:teuthology.orchestra.run.smithi105.stdout:Upgraded: 2024-03-21T08:46:02.552 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:02.552 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:02.552 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.553 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout:Installed: 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:46:02.554 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.555 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:46:02.556 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:46:02.557 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:46:02.557 INFO:teuthology.orchestra.run.smithi105.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:46:02.557 INFO:teuthology.orchestra.run.smithi105.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:46:02.557 INFO:teuthology.orchestra.run.smithi105.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:46:02.557 INFO:teuthology.orchestra.run.smithi105.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:46:02.557 INFO:teuthology.orchestra.run.smithi105.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:46:02.558 INFO:teuthology.orchestra.run.smithi105.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:46:02.559 INFO:teuthology.orchestra.run.smithi105.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:46:02.560 INFO:teuthology.orchestra.run.smithi105.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:46:02.561 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:46:02.562 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:46:02.563 INFO:teuthology.orchestra.run.smithi105.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:46:02.564 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:46:02.565 INFO:teuthology.orchestra.run.smithi105.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:46:02.566 INFO:teuthology.orchestra.run.smithi105.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:46:02.567 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:46:02.567 INFO:teuthology.orchestra.run.smithi105.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:46:02.567 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:46:02.567 INFO:teuthology.orchestra.run.smithi105.stdout:Complete! 2024-03-21T08:46:02.698 INFO:teuthology.orchestra.run.smithi087.stdout: Running scriptlet: elfutils-libelf-0.189-3.el9.x86_64 198/198 2024-03-21T08:46:02.698 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 1/198 2024-03-21T08:46:02.698 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2/198 2024-03-21T08:46:02.698 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 3/198 2024-03-21T08:46:02.698 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 4/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2231.g2edfe 5/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 6/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 7/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 8/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 9/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 10/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 11/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 12/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 13/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 14/198 2024-03-21T08:46:02.699 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_ 15/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 16/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 17/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_6 18/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86 19/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 20/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el 21/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ceph-common-2:19.0.0-2231.g2edfee68.el9. 22/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_6 23/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 24/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 25/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 26/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 27/198 2024-03-21T08:46:02.700 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 28/198 2024-03-21T08:46:02.701 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 29/198 2024-03-21T08:46:02.701 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68. 30/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noa 31/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.n 32/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2231.g2ed 33/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el 34/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 35/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.e 36/198 2024-03-21T08:46:02.717 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 37/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 38/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 39/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flex-2.6.4-9.el9.x86_64 43/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 44/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 45/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 46/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 47/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 48/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 49/198 2024-03-21T08:46:02.718 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 50/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 51/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 52/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 53/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-5.4.4-4.el9.x86_64 54/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 55/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 56/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 57/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-13.11-1.el9.x86_64 58/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 59/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 60/198 2024-03-21T08:46:02.719 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 61/198 2024-03-21T08:46:02.720 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 62/198 2024-03-21T08:46:02.720 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 63/198 2024-03-21T08:46:02.720 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 64/198 2024-03-21T08:46:02.720 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 65/198 2024-03-21T08:46:02.720 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 66/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 67/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 68/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 69/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 70/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 71/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 72/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 73/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 74/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/198 2024-03-21T08:46:02.721 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 84/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 85/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 86/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 87/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 88/198 2024-03-21T08:46:02.722 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-ply-3.11-14.el9.noarch 89/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 90/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 91/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : jq-1.6-16.el9.x86_64 92/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 93/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 94/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 95/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 96/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 97/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 98/198 2024-03-21T08:46:02.723 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 99/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 100/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-23.11.0-1.el9.x86_64 101/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatlib-service-23.11.0-1.el9.x86_64 102/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : qatzip-libs-1.1.2-2.el9.x86_64 103/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 104/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 108/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 110/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 111/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/198 2024-03-21T08:46:02.724 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 114/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 115/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 119/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 123/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 124/198 2024-03-21T08:46:02.725 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 125/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 126/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 127/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 128/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 129/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 130/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 131/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 132/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 133/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 134/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 135/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 136/198 2024-03-21T08:46:02.726 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 137/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 138/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 139/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 140/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 141/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 142/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 143/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 144/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 145/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 146/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 147/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 148/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : re2-1:20211101-3.el9.x86_64 149/198 2024-03-21T08:46:02.727 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 150/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 151/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 152/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 153/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 154/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 155/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 156/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.47.2-1.el9.x86_64 157/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 158/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.47.2-1.el9.x86_64 159/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 160/198 2024-03-21T08:46:02.728 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.47.2-1.el9.x86_64 161/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 162/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.47.2-1.el9.x86_64 163/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 164/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : acl-2.3.1-4.el9.x86_64 165/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : acl-2.3.1-3.el9.x86_64 166/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-debuginfod-client-0.190-2.el9.x86_64 167/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 168/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-0.190-2.el9.x86_64 169/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 170/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libs-0.190-2.el9.x86_64 171/198 2024-03-21T08:46:02.729 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 172/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 173/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 174/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libblkid-2.37.4-18.el9.x86_64 175/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libblkid-2.37.4-15.el9.x86_64 176/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libfdisk-2.37.4-18.el9.x86_64 177/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libfdisk-2.37.4-15.el9.x86_64 178/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libmount-2.37.4-18.el9.x86_64 179/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libmount-2.37.4-15.el9.x86_64 180/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libsmartcols-2.37.4-18.el9.x86_64 181/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libsmartcols-2.37.4-15.el9.x86_64 182/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-2.37.4-18.el9.x86_64 183/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libuuid-2.37.4-15.el9.x86_64 184/198 2024-03-21T08:46:02.730 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 185/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 186/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 187/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 188/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : util-linux-2.37.4-18.el9.x86_64 189/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : util-linux-2.37.4-15.el9.x86_64 190/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : util-linux-core-2.37.4-18.el9.x86_64 191/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : util-linux-core-2.37.4-15.el9.x86_64 192/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : util-linux-user-2.37.4-18.el9.x86_64 193/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : util-linux-user-2.37.4-15.el9.x86_64 194/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-6.3.0-1.el9.x86_64 195/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : xfsprogs-5.19.0-4.el9.x86_64 196/198 2024-03-21T08:46:02.731 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 197/198 2024-03-21T08:46:03.048 DEBUG:teuthology.parallel:result is None 2024-03-21T08:46:05.490 INFO:teuthology.orchestra.run.smithi087.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 198/198 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout:Upgraded: 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-libnm-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-team-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: NetworkManager-tui-1:1.47.2-1.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: acl-2.3.1-4.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-debuginfod-client-0.190-2.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-0.190-2.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libs-0.190-2.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: libacl-2.3.1-4.el9.x86_64 2024-03-21T08:46:05.491 INFO:teuthology.orchestra.run.smithi087.stdout: libacl-devel-2.3.1-4.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: libblkid-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: libfdisk-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: libmount-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: librados2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: librbd1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: libsmartcols-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-1:3.0.7-27.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-libs-1:3.0.7-27.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-core-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: util-linux-user-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: xfsdump-3.1.12-4.el9.x86_64 2024-03-21T08:46:05.492 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-6.3.0-1.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout:Installed: 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: automake-1.16.2-8.el9.noarch 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: bison-3.7.4-5.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-base-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-grafana-dashboards-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-immutable-object-cache-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mds-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.493 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-dashboard-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-modules-core-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mgr-rook-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-mon-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-osd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-prometheus-alerts-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-radosgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-selinux-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.494 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-test-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: ceph-volume-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: cephadm-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-shell-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: cephfs-top-2:19.0.0-2231.g2edfee68.el9.noarch 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: flex-2.6.4-9.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: fuse-2.9.9-15.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-03-21T08:46:05.495 INFO:teuthology.orchestra.run.smithi087.stdout: grpc-data-1.46.7-10.el9.noarch 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: jq-1.6-16.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-9.0.0-10.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libcephfs2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libcephsqlite-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: liboath-2.6.7-2.el9.x86_64 2024-03-21T08:46:05.496 INFO:teuthology.orchestra.run.smithi087.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: librados-devel-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: libradosstriper1-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: librgw2-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: libtool-2.4.6-45.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: libunwind-1.6.2-1.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-03-21T08:46:05.497 INFO:teuthology.orchestra.run.smithi087.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: lua-5.4.4-4.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: luarocks-3.9.2-1.el9.noarch 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-0.3.21-2.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-13.11-1.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-contrib-13.11-1.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: postgresql-server-13.11-1.el9.x86_64 2024-03-21T08:46:05.498 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-3.14.0-13.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-babel-2.9.1-2.el9.noarch 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-argparse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ceph-common-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cephfs-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-03-21T08:46:05.499 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-devel-3.9.17-2.el9.x86_64 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-03-21T08:46:05.500 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-mako-1.1.4-6.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-packaging-20.9-5.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-ply-3.11-14.el9.noarch 2024-03-21T08:46:05.501 INFO:teuthology.orchestra.run.smithi087.stdout: python3-portend-3.1.0-2.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pycparser-2.20-6.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-pytz-2021.1-5.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rados-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-2.25.1-8.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-03-21T08:46:05.502 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rgw-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-routes-2.5.1-5.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-rsa-4.9-2.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-toml-0.10.2-6.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-webob-1.8.7-6.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-23.11.0-1.el9.x86_64 2024-03-21T08:46:05.503 INFO:teuthology.orchestra.run.smithi087.stdout: qatlib-service-23.11.0-1.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: qatzip-libs-1.1.2-2.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-fuse-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-mirror-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: rbd-nbd-2:19.0.0-2231.g2edfee68.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: re2-1:20211101-3.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: socat-1.7.4.1-5.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: thrift-0.15.0-2.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: uuid-1.6.2-55.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-03-21T08:46:05.504 INFO:teuthology.orchestra.run.smithi087.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-03-21T08:46:05.505 INFO:teuthology.orchestra.run.smithi087.stdout: zlib-devel-1.2.11-40.el9.x86_64 2024-03-21T08:46:05.505 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:46:05.505 INFO:teuthology.orchestra.run.smithi087.stdout:Complete! 2024-03-21T08:46:06.002 DEBUG:teuthology.parallel:result is None 2024-03-21T08:46:06.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:06.212 DEBUG:teuthology.orchestra.run.smithi087:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:46:06.234 INFO:teuthology.orchestra.run.smithi087.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:46:06.234 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:46:06.234 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:46:06.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:06.376 DEBUG:teuthology.orchestra.run.smithi100:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:46:06.401 INFO:teuthology.orchestra.run.smithi100.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:46:06.401 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:46:06.401 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:46:06.403 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:06.541 DEBUG:teuthology.orchestra.run.smithi105:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-03-21T08:46:06.567 INFO:teuthology.orchestra.run.smithi105.stdout:19.0.0-2231.g2edfee68.el9 2024-03-21T08:46:06.567 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2231.g2edfee68.el9 2024-03-21T08:46:06.567 INFO:teuthology.task.install:The correct ceph version 19.0.0-2231.g2edfee68 is installed. 2024-03-21T08:46:06.569 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-21T08:46:06.569 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:46:06.569 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:46:06.600 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:46:06.600 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:46:06.635 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:46:06.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-21T08:46:06.669 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-21T08:46:06.670 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:46:06.670 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:46:06.698 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:46:06.765 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:46:06.765 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:46:06.796 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:46:06.867 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:46:06.867 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-03-21T08:46:06.899 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-21T08:46:06.969 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-21T08:46:06.969 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:46:06.969 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:46:06.997 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:46:07.064 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:46:07.064 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:46:07.093 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:46:07.161 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:46:07.161 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-21T08:46:07.192 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-21T08:46:07.263 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-21T08:46:07.263 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:46:07.263 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:46:07.291 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:46:07.360 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:46:07.360 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:46:07.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:46:07.454 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:46:07.454 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-03-21T08:46:07.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-21T08:46:07.559 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-21T08:46:07.658 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-21T08:46:07.658 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:07.662 INFO:tasks.cephadm:Cluster fsid is 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:46:07.662 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-21T08:46:07.662 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.87', 'mon.b': '172.21.15.100', 'mon.c': '172.21.15.105'} 2024-03-21T08:46:07.662 INFO:tasks.cephadm:First mon is mon.a on smithi087 2024-03-21T08:46:07.662 INFO:tasks.cephadm:First mgr is x 2024-03-21T08:46:07.662 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-21T08:46:07.663 DEBUG:teuthology.orchestra.run.smithi087:> sudo hostname $(hostname -s) 2024-03-21T08:46:07.692 DEBUG:teuthology.orchestra.run.smithi100:> sudo hostname $(hostname -s) 2024-03-21T08:46:07.720 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-03-21T08:46:07.751 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-03-21T08:46:07.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:07.971 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '2edfee68ad7243bed422a049b6f5f58619300f11', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/77791/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2231-g2edfee68', 'node_name': '172.21.5.34+adami04', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2231.g2edfee68'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-03-20 07:25:08.579979', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-vshankar-testing-20240320.053002', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-03-21T08:46:08.158 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-vshankar-testing-20240320.053002, sha1 2edfee68ad7243bed422a049b6f5f58619300f11 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:46:08.160 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm 2024-03-21T08:46:08.161 DEBUG:teuthology.orchestra.run.smithi087:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:08.760 INFO:teuthology.orchestra.run.smithi087.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:46 /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:08.761 DEBUG:teuthology.orchestra.run.smithi100:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.190 INFO:teuthology.orchestra.run.smithi100.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:46 /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.292 DEBUG:teuthology.orchestra.run.smithi105:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-vshankar-testing-20240320.053002/2edfee68ad7243bed422a049b6f5f58619300f11/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.743 INFO:teuthology.orchestra.run.smithi105.stdout:-rw-r--r--. 1 ubuntu ubuntu 769188 Mar 21 08:46 /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.743 DEBUG:teuthology.orchestra.run.smithi087:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.762 DEBUG:teuthology.orchestra.run.smithi100:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.781 DEBUG:teuthology.orchestra.run.smithi105:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-03-21T08:46:09.817 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 on all hosts... 2024-03-21T08:46:09.817 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:46:09.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:46:09.824 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 pull 2024-03-21T08:46:10.659 INFO:teuthology.orchestra.run.smithi100.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:46:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:46:10.697 INFO:teuthology.orchestra.run.smithi087.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:46:40.750 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:46:40.751 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:46:40.751 INFO:teuthology.orchestra.run.smithi087.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:46:40.751 INFO:teuthology.orchestra.run.smithi087.stdout: "repo_digests": [ 2024-03-21T08:46:40.751 INFO:teuthology.orchestra.run.smithi087.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:46:40.751 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T08:46:40.752 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout: "repo_digests": [ 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout: ] 2024-03-21T08:46:41.841 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_version": "ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev)", 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: "image_id": "237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54", 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: "repo_digests": [ 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232" 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout: ] 2024-03-21T08:46:43.697 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T08:46:43.719 DEBUG:teuthology.orchestra.run.smithi087:> sudo mkdir -p /etc/ceph 2024-03-21T08:46:43.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo mkdir -p /etc/ceph 2024-03-21T08:46:43.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-03-21T08:46:43.818 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 777 /etc/ceph 2024-03-21T08:46:43.849 DEBUG:teuthology.orchestra.run.smithi100:> sudo chmod 777 /etc/ceph 2024-03-21T08:46:43.875 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-03-21T08:46:43.906 INFO:tasks.cephadm:Writing seed config... 2024-03-21T08:46:43.907 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [client] debug client = 20 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-03-21T08:46:43.908 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-03-21T08:46:43.909 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-21T08:46:43.910 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-03-21T08:46:43.911 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:46:43.911 DEBUG:teuthology.orchestra.run.smithi087:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-21T08:46:43.928 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 75a2f1f0-e75f-11ee-95cd-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-03-21T08:46:43.928 DEBUG:teuthology.orchestra.run.smithi087:mon.a> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a.service 2024-03-21T08:46:43.971 DEBUG:teuthology.orchestra.run.smithi087:mgr.x> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service 2024-03-21T08:46:44.016 INFO:tasks.cephadm:Bootstrapping... 2024-03-21T08:46:44.016 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 -v bootstrap --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.87 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:46:44.207 INFO:teuthology.orchestra.run.smithi087.stdout:-------------------------------------------------------------------------------- 2024-03-21T08:46:44.208 INFO:teuthology.orchestra.run.smithi087.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11', '-v', 'bootstrap', '--fsid', '75a2f1f0-e75f-11ee-95cd-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.87', '--skip-admin-label'] 2024-03-21T08:46:44.208 INFO:teuthology.orchestra.run.smithi087.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-21T08:46:44.208 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying podman|docker is present... 2024-03-21T08:46:44.233 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T08:46:44.234 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying lvm2 is present... 2024-03-21T08:46:44.234 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying time synchronization is in place... 2024-03-21T08:46:44.241 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T08:46:44.241 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T08:46:44.247 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T08:46:44.247 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-03-21T08:46:44.253 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout enabled 2024-03-21T08:46:44.259 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout active 2024-03-21T08:46:44.259 INFO:teuthology.orchestra.run.smithi087.stdout:Unit chronyd.service is enabled and running 2024-03-21T08:46:44.259 INFO:teuthology.orchestra.run.smithi087.stdout:Repeating the final host check... 2024-03-21T08:46:44.283 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 4.9.3-dev 2024-03-21T08:46:44.283 INFO:teuthology.orchestra.run.smithi087.stdout:podman (/bin/podman) version 4.9.3 is present 2024-03-21T08:46:44.283 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl is present 2024-03-21T08:46:44.283 INFO:teuthology.orchestra.run.smithi087.stdout:lvcreate is present 2024-03-21T08:46:44.289 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-21T08:46:44.289 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-21T08:46:44.295 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-21T08:46:44.295 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-03-21T08:46:44.301 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout enabled 2024-03-21T08:46:44.307 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout active 2024-03-21T08:46:44.307 INFO:teuthology.orchestra.run.smithi087.stdout:Unit chronyd.service is enabled and running 2024-03-21T08:46:44.307 INFO:teuthology.orchestra.run.smithi087.stdout:Host looks OK 2024-03-21T08:46:44.307 INFO:teuthology.orchestra.run.smithi087.stdout:Cluster fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:46:44.307 INFO:teuthology.orchestra.run.smithi087.stdout:Acquiring lock 140556910445520 on /run/cephadm/75a2f1f0-e75f-11ee-95cd-87774f69a715.lock 2024-03-21T08:46:44.308 INFO:teuthology.orchestra.run.smithi087.stdout:Lock 140556910445520 acquired on /run/cephadm/75a2f1f0-e75f-11ee-95cd-87774f69a715.lock 2024-03-21T08:46:44.308 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying IP 172.21.15.87 port 3300 ... 2024-03-21T08:46:44.308 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying IP 172.21.15.87 port 6789 ... 2024-03-21T08:46:44.308 INFO:teuthology.orchestra.run.smithi087.stdout:Base mon IP(s) is [172.21.15.87:3300, 172.21.15.87:6789], mon addrv is [v2:172.21.15.87:3300,v1:172.21.15.87:6789] 2024-03-21T08:46:44.311 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.87 metric 100 2024-03-21T08:46:44.311 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.87 metric 100 2024-03-21T08:46:44.314 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-21T08:46:44.314 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-21T08:46:44.314 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 1024 expires 1786sec hoplimit 64 pref medium 2024-03-21T08:46:44.316 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-21T08:46:44.316 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-21T08:46:44.316 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T08:46:44.316 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-21T08:46:44.316 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-03-21T08:46:44.317 INFO:teuthology.orchestra.run.smithi087.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-21T08:46:44.317 INFO:teuthology.orchestra.run.smithi087.stdout:Mon IP `172.21.15.87` is in CIDR network `172.21.0.0/20` 2024-03-21T08:46:44.317 INFO:teuthology.orchestra.run.smithi087.stdout:Mon IP `172.21.15.87` is in CIDR network `172.21.0.0/20` 2024-03-21T08:46:44.317 INFO:teuthology.orchestra.run.smithi087.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-03-21T08:46:44.318 INFO:teuthology.orchestra.run.smithi087.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-21T08:46:44.319 INFO:teuthology.orchestra.run.smithi087.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:46:44.827 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stdout 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T08:46:44.828 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11... 2024-03-21T08:46:44.828 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Getting image source signatures 2024-03-21T08:46:44.828 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying blob sha256:96a207f569d15e36f8e9e3b2feafd8af7b93861f24c2b60ada6c9667dd5405be 2024-03-21T08:46:44.828 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying blob sha256:9516588f908cdb14b5e4062a6063db2d9291b355c3f32e2a99864608a4a7b649 2024-03-21T08:46:44.828 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Copying config sha256:237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 2024-03-21T08:46:44.828 INFO:teuthology.orchestra.run.smithi087.stdout:/bin/podman: stderr Writing manifest to image destination 2024-03-21T08:46:45.657 INFO:teuthology.orchestra.run.smithi087.stdout:ceph: stdout ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T08:46:45.657 INFO:teuthology.orchestra.run.smithi087.stdout:Ceph version: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev) 2024-03-21T08:46:45.657 INFO:teuthology.orchestra.run.smithi087.stdout:Extracting ceph user uid/gid from container image... 2024-03-21T08:46:46.561 INFO:teuthology.orchestra.run.smithi087.stdout:stat: stdout 167 167 2024-03-21T08:46:46.562 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial keys... 2024-03-21T08:46:47.501 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQD28/tljzRxNhAAurgXokQ4Z+iDVkBxot/ZVw== 2024-03-21T08:46:54.208 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQD38/tlCu8MNhAAI60YdvfgYiCJmZzeJBvsTw== 2024-03-21T08:47:00.805 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph-authtool: stdout AQAA9Ptlw9GdNBAA9eUZb4GkYjY+zPeJvQZpAQ== 2024-03-21T08:47:00.806 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial monmap... 2024-03-21T08:47:02.345 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:monmaptool for a [v2:172.21.15.87:3300,v1:172.21.15.87:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:setting min_mon_release = quincy 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: set fsid to 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-21T08:47:02.346 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:47:02.347 INFO:teuthology.orchestra.run.smithi087.stdout:Creating mon... 2024-03-21T08:47:03.259 INFO:teuthology.orchestra.run.smithi087.stdout:create mon.a on 2024-03-21T08:47:03.458 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-03-21T08:47:03.616 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-03-21T08:47:03.805 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715.target. 2024-03-21T08:47:03.806 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715.target → /etc/systemd/system/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715.target. 2024-03-21T08:47:04.049 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a 2024-03-21T08:47:04.049 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to reset failed state of unit ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a.service: Unit ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a.service not loaded. 2024-03-21T08:47:04.209 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715.target.wants/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a.service → /etc/systemd/system/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@.service. 2024-03-21T08:47:04.564 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 podman[31992]: 2024-03-21 08:47:04.461050761 +0000 UTC m=+0.119599752 container create 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T08:47:04.740 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:47:04.741 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-03-21T08:47:04.747 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:47:04.747 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-03-21T08:47:04.748 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-03-21T08:47:04.748 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T08:47:04.748 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mon to start... 2024-03-21T08:47:04.748 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mon... 2024-03-21T08:47:04.845 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 podman[31992]: 2024-03-21 08:47:04.65502558 +0000 UTC m=+0.313574570 container init 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-03-21T08:47:04.845 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 podman[31992]: 2024-03-21 08:47:04.658296494 +0000 UTC m=+0.316845487 container start 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2) 2024-03-21T08:47:04.845 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 ceph-mon[32011]: mkfs 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:04.845 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 ceph-mon[32011]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:47:04.845 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 bash[31992]: 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 2024-03-21T08:47:04.845 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:04 smithi087 systemd[1]: Started Ceph mon.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:47:05.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:05 smithi087 ceph-mon[32011]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:47:05.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:05 smithi087 ceph-mon[32011]: monmap e1: 1 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:47:05.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:05 smithi087 ceph-mon[32011]: fsmap 2024-03-21T08:47:05.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:05 smithi087 ceph-mon[32011]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T08:47:05.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:05 smithi087 ceph-mon[32011]: mgrmap e1: no daemons active 2024-03-21T08:47:05.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:05 smithi087 ceph-mon[32011]: from='client.? 172.21.15.87:0/2851051516' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-21T08:47:06.121 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout cluster: 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout id: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout services: 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.564589s) 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout data: 2024-03-21T08:47:06.128 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout pgs: 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.204+0000 7f9a27afc700 1 Processor -- start 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.204+0000 7f9a27afc700 1 -- start start 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.204+0000 7f9a27afc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2010a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.204+0000 7f9a27afc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2007be30 con 0x7f9a20108580 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.205+0000 7f9a25898700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2010a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.205+0000 7f9a25898700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2010a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57226/0 (socket says 172.21.15.87:57226) 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.205+0000 7f9a25898700 1 -- 172.21.15.87:0/2433741788 learned_addr learned my addr 172.21.15.87:0/2433741788 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.205+0000 7f9a25898700 1 -- 172.21.15.87:0/2433741788 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2010aeb0 con 0x7f9a20108580 2024-03-21T08:47:06.129 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.205+0000 7f9a25898700 1 --2- 172.21.15.87:0/2433741788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2010a970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be9841db01b8eaff server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.205+0000 7f9a24896700 1 -- 172.21.15.87:0/2433741788 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c004160 con 0x7f9a20108580 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a24896700 1 -- 172.21.15.87:0/2433741788 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c004300 con 0x7f9a20108580 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a24896700 1 -- 172.21.15.87:0/2433741788 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c0036a0 con 0x7f9a20108580 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 -- 172.21.15.87:0/2433741788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 msgr2=0x7f9a2010a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 --2- 172.21.15.87:0/2433741788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2010a970 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c0098a0 tx=0x7f9a1c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 -- 172.21.15.87:0/2433741788 shutdown_connections 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 --2- 172.21.15.87:0/2433741788 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2010a970 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 -- 172.21.15.87:0/2433741788 >> 172.21.15.87:0/2433741788 conn(0x7f9a20101f10 msgr2=0x7f9a20104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 -- 172.21.15.87:0/2433741788 shutdown_connections 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.206+0000 7f9a27afc700 1 -- 172.21.15.87:0/2433741788 wait complete. 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a27afc700 1 Processor -- start 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a27afc700 1 -- start start 2024-03-21T08:47:06.130 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a27afc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a27afc700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2010b2c0 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a25898700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a25898700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2019b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57232/0 (socket says 172.21.15.87:57232) 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a25898700 1 -- 172.21.15.87:0/2851051516 learned_addr learned my addr 172.21.15.87:0/2851051516 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.207+0000 7f9a25898700 1 -- 172.21.15.87:0/2851051516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2019b750 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a25898700 1 --2- 172.21.15.87:0/2851051516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2019b210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c000c00 tx=0x7f9a1c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c003db0 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a1c011630 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9a1c011950 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2019b9e0 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2019e680 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f9a1c01a460 con 0x7f9a20108580 2024-03-21T08:47:06.131 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.208+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9a1c01b780 con 0x7f9a20108580 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.209+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a20069250 con 0x7f9a20108580 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.211+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9a1c01b980 con 0x7f9a20108580 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.282+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f9a2002dc80 con 0x7f9a20108580 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a16ffd700 1 -- 172.21.15.87:0/2851051516 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f9a1c01bb60 con 0x7f9a20108580 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 msgr2=0x7f9a2019b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 --2- 172.21.15.87:0/2851051516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2019b210 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f9a1c000c00 tx=0x7f9a1c004750 comp rx=0 tx=0).stop 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 shutdown_connections 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 --2- 172.21.15.87:0/2851051516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9a20108580 0x7f9a2019b210 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 >> 172.21.15.87:0/2851051516 conn(0x7f9a20101f10 msgr2=0x7f9a20104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 shutdown_connections 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:05.283+0000 7f9a27afc700 1 -- 172.21.15.87:0/2851051516 wait complete. 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:mon is available 2024-03-21T08:47:06.132 INFO:teuthology.orchestra.run.smithi087.stdout:Assimilating anything we can from ceph.conf... 2024-03-21T08:47:07.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:06 smithi087 ceph-mon[32011]: from='client.? 172.21.15.87:0/1970518315' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T08:47:07.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:06 smithi087 ceph-mon[32011]: from='client.? 172.21.15.87:0/1970518315' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout fsid = 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.87:3300,v1:172.21.15.87:6789] 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:07.327 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.543+0000 7fba93953700 1 Processor -- start 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.543+0000 7fba93953700 1 -- start start 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.543+0000 7fba93953700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.543+0000 7fba93953700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c108540 con 0x7fba8c107b50 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba916ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:07.328 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba916ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57236/0 (socket says 172.21.15.87:57236) 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba916ef700 1 -- 172.21.15.87:0/3922386745 learned_addr learned my addr 172.21.15.87:0/3922386745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba916ef700 1 -- 172.21.15.87:0/3922386745 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c108d00 con 0x7fba8c107b50 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba916ef700 1 --2- 172.21.15.87:0/3922386745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c107f70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba880098a0 tx=0x7fba88009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5bdfed3628b5ad server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba83fff700 1 -- 172.21.15.87:0/3922386745 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba88004160 con 0x7fba8c107b50 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba83fff700 1 -- 172.21.15.87:0/3922386745 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88004300 con 0x7fba8c107b50 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.544+0000 7fba83fff700 1 -- 172.21.15.87:0/3922386745 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba880036a0 con 0x7fba8c107b50 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 -- 172.21.15.87:0/3922386745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 msgr2=0x7fba8c107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 --2- 172.21.15.87:0/3922386745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c107f70 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fba880098a0 tx=0x7fba88009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 -- 172.21.15.87:0/3922386745 shutdown_connections 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 --2- 172.21.15.87:0/3922386745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c107f70 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 -- 172.21.15.87:0/3922386745 >> 172.21.15.87:0/3922386745 conn(0x7fba8c103130 msgr2=0x7fba8c105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 -- 172.21.15.87:0/3922386745 shutdown_connections 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 -- 172.21.15.87:0/3922386745 wait complete. 2024-03-21T08:47:07.329 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 Processor -- start 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.545+0000 7fba93953700 1 -- start start 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba93953700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c07db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba93953700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba8c109230 con 0x7fba8c107b50 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba916ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c07db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba916ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c07db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:57252/0 (socket says 172.21.15.87:57252) 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba916ef700 1 -- 172.21.15.87:0/1970518315 learned_addr learned my addr 172.21.15.87:0/1970518315 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba916ef700 1 -- 172.21.15.87:0/1970518315 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c07e0d0 con 0x7fba8c107b50 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba916ef700 1 --2- 172.21.15.87:0/1970518315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c07db90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba88004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba88003db0 con 0x7fba8c107b50 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88011630 con 0x7fba8c107b50 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba8c07c2c0 con 0x7fba8c107b50 2024-03-21T08:47:07.330 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba88011950 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba8c07c800 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fba88011af0 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.546+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba8801c450 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.547+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba8c069250 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.549+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fba88032020 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.615+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fba8c194850 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.618+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fba8c194850 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.619+0000 7fba827fc700 1 -- 172.21.15.87:0/1970518315 <== mon.0 v2:172.21.15.87:3300/0 8 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fba88004260 con 0x7fba8c107b50 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.619+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 msgr2=0x7fba8c07db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.619+0000 7fba93953700 1 --2- 172.21.15.87:0/1970518315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c07db90 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fba88000c00 tx=0x7fba88004750 comp rx=0 tx=0).stop 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.619+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 shutdown_connections 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.619+0000 7fba93953700 1 --2- 172.21.15.87:0/1970518315 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fba8c107b50 0x7fba8c07db90 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.619+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 >> 172.21.15.87:0/1970518315 conn(0x7fba8c103130 msgr2=0x7fba8c191f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.620+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 shutdown_connections 2024-03-21T08:47:07.331 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:06.620+0000 7fba93953700 1 -- 172.21.15.87:0/1970518315 wait complete. 2024-03-21T08:47:07.332 INFO:teuthology.orchestra.run.smithi087.stdout:Generating new minimal ceph.conf... 2024-03-21T08:47:07.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:07 smithi087 ceph-mon[32011]: from='client.? 172.21.15.87:0/3829860172' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0caee0c700 1 Processor -- start 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0caee0c700 1 -- start start 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0caee0c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca81084a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0caee0c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8108a70 con 0x7f0ca8108080 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0cacba8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca81084a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0cacba8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca81084a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45282/0 (socket says 172.21.15.87:45282) 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0cacba8700 1 -- 172.21.15.87:0/2044126269 learned_addr learned my addr 172.21.15.87:0/2044126269 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0cacba8700 1 -- 172.21.15.87:0/2044126269 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca8105b80 con 0x7f0ca8108080 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.715+0000 7f0cacba8700 1 --2- 172.21.15.87:0/2044126269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca81084a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0098a0 tx=0x7f0c9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae893c82ada50571 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:08.356 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0ca77fe700 1 -- 172.21.15.87:0/2044126269 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c004160 con 0x7f0ca8108080 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0ca77fe700 1 -- 172.21.15.87:0/2044126269 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c9c004300 con 0x7f0ca8108080 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0ca77fe700 1 -- 172.21.15.87:0/2044126269 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c011b50 con 0x7f0ca8108080 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 -- 172.21.15.87:0/2044126269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 msgr2=0x7f0ca81084a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 --2- 172.21.15.87:0/2044126269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca81084a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c0098a0 tx=0x7f0c9c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 -- 172.21.15.87:0/2044126269 shutdown_connections 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 --2- 172.21.15.87:0/2044126269 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca81084a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 -- 172.21.15.87:0/2044126269 >> 172.21.15.87:0/2044126269 conn(0x7f0ca8101740 msgr2=0x7f0ca8103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 -- 172.21.15.87:0/2044126269 shutdown_connections 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.716+0000 7f0caee0c700 1 -- 172.21.15.87:0/2044126269 wait complete. 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0caee0c700 1 Processor -- start 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0caee0c700 1 -- start start 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0caee0c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:08.357 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0caee0c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca8106c00 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0cacba8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0cacba8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca819dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45284/0 (socket says 172.21.15.87:45284) 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0cacba8700 1 -- 172.21.15.87:0/3829860172 learned_addr learned my addr 172.21.15.87:0/3829860172 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0cacba8700 1 -- 172.21.15.87:0/3829860172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ca819e1d0 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.717+0000 7f0cacba8700 1 --2- 172.21.15.87:0/3829860172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca819dc90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009fb0 tx=0x7f0c9c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c0038b0 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c9c011e50 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca819e460 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0c9c019420 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca81a1150 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f0c9c019ad0 con 0x7f0ca8108080 2024-03-21T08:47:08.358 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.718+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0c9c024730 con 0x7f0ca8108080 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.719+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca8069250 con 0x7f0ca8108080 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.721+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f0c9c043dd0 con 0x7f0ca8108080 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.790+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f0ca8197290 con 0x7f0ca8108080 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.791+0000 7f0ca5ffb700 1 -- 172.21.15.87:0/3829860172 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f0c9c032030 con 0x7f0ca8108080 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.791+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 msgr2=0x7f0ca819dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.791+0000 7f0caee0c700 1 --2- 172.21.15.87:0/3829860172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca819dc90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c009fb0 tx=0x7f0c9c011bf0 comp rx=0 tx=0).stop 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.791+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 shutdown_connections 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.791+0000 7f0caee0c700 1 --2- 172.21.15.87:0/3829860172 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0ca8108080 0x7f0ca819dc90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.791+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 >> 172.21.15.87:0/3829860172 conn(0x7f0ca8101740 msgr2=0x7f0ca8194370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.792+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 shutdown_connections 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:07.792+0000 7f0caee0c700 1 -- 172.21.15.87:0/3829860172 wait complete. 2024-03-21T08:47:08.359 INFO:teuthology.orchestra.run.smithi087.stdout:Restarting the monitor... 2024-03-21T08:47:08.627 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:08 smithi087 systemd[1]: Stopping Ceph mon.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:47:08.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a[32007]: 2024-03-21T08:47:08.628+0000 7f36fec22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T08:47:08.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a[32007]: 2024-03-21T08:47:08.628+0000 7f36fec22700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-21T08:47:08.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:08 smithi087 podman[32257]: 2024-03-21 08:47:08.71091356 +0000 UTC m=+0.191626031 container died 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-03-21T08:47:09.243 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:08 smithi087 podman[32257]: 2024-03-21 08:47:08.995421471 +0000 UTC m=+0.476133942 container cleanup 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:47:09.244 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:08 smithi087 bash[32257]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a 2024-03-21T08:47:09.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 podman[32271]: 2024-03-21 08:47:09.246350268 +0000 UTC m=+0.530586325 container remove 350ea9f189a031c0f1feea32760276697e0d26e7e796cc225a2040f042680889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T08:47:09.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-21T08:47:09.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 systemd[1]: Stopped Ceph mon.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:47:09.580 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 systemd[1]: Starting Ceph mon.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:47:09.830 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 podman[32366]: 2024-03-21 08:47:09.487110419 +0000 UTC m=+0.020135411 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:47:09.830 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 podman[32366]: 2024-03-21 08:47:09.598129455 +0000 UTC m=+0.131154421 container create b1c901b5acae9f9a9cb3758fd5d36c814c02e635c57c0dac22dc45dd8623a752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 podman[32366]: 2024-03-21 08:47:09.799614113 +0000 UTC m=+0.332639100 container init b1c901b5acae9f9a9cb3758fd5d36c814c02e635c57c0dac22dc45dd8623a752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 podman[32366]: 2024-03-21 08:47:09.80255655 +0000 UTC m=+0.335581516 container start b1c901b5acae9f9a9cb3758fd5d36c814c02e635c57c0dac22dc45dd8623a752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: set uid:gid to 167:167 (ceph:ceph) 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: ceph version 19.0.0-2231-g2edfee68 (2edfee68ad7243bed422a049b6f5f58619300f11) squid (dev), process ceph-mon, pid 2 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: pidfile_write: ignore empty --pid-file 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: load: jerasure load: lrc 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: RocksDB version: 7.9.2 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Git sha 0 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Compile date 2024-03-20 06:23:48 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: DB SUMMARY 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: DB Session ID: JQXRSWCP7ZWUTO9R9P45 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: CURRENT file: CURRENT 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: IDENTITY file: IDENTITY 2024-03-21T08:47:09.831 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-21T08:47:09.832 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-21T08:47:09.832 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100194 ; 2024-03-21T08:47:09.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.error_if_exists: 0 2024-03-21T08:47:09.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.create_if_missing: 0 2024-03-21T08:47:09.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.paranoid_checks: 1 2024-03-21T08:47:09.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.flush_verify_memtable_count: 1 2024-03-21T08:47:09.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.env: 0x55c96fcc5b20 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.fs: PosixFileSystem 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.info_log: 0x55c9722f1920 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_file_opening_threads: 16 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.statistics: (nil) 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.use_fsync: 0 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_log_file_size: 0 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.log_file_time_to_roll: 0 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.keep_log_file_num: 1000 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.recycle_log_file_num: 0 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.allow_fallocate: 1 2024-03-21T08:47:09.835 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.allow_mmap_reads: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.allow_mmap_writes: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.use_direct_reads: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.create_missing_column_families: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.db_log_dir: 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.wal_dir: 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.table_cache_numshardbits: 6 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.WAL_ttl_seconds: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.WAL_size_limit_MB: 0 2024-03-21T08:47:09.836 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.is_fd_close_on_exec: 1 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.advise_random_on_open: 1 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.db_write_buffer_size: 0 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.write_buffer_manager: 0x55c9722c41e0 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.use_adaptive_mutex: 0 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.rate_limiter: (nil) 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-21T08:47:09.837 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.wal_recovery_mode: 2 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.enable_thread_tracking: 0 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.enable_pipelined_write: 0 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.unordered_write: 0 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-21T08:47:09.838 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.row_cache: None 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.wal_filter: None 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.allow_ingest_behind: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.two_write_queues: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.manual_wal_flush: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.wal_compression: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.atomic_flush: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.persist_stats_to_disk: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.write_dbid_to_manifest: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.log_readahead_size: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.best_efforts_recovery: 0 2024-03-21T08:47:09.839 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.allow_data_in_errors: 0 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.db_host_id: __hostname__ 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.enforce_single_del_contracts: true 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_background_jobs: 2 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_background_compactions: -1 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_subcompactions: 1 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.delayed_write_rate : 16777216 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_total_wal_size: 0 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.stats_dump_period_sec: 600 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.stats_persist_period_sec: 600 2024-03-21T08:47:09.840 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-21T08:47:09.841 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_open_files: -1 2024-03-21T08:47:09.841 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bytes_per_sync: 0 2024-03-21T08:47:09.841 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.wal_bytes_per_sync: 0 2024-03-21T08:47:09.842 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.strict_bytes_per_sync: 0 2024-03-21T08:47:09.842 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_readahead_size: 0 2024-03-21T08:47:09.842 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_background_flushes: -1 2024-03-21T08:47:09.842 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Compression algorithms supported: 2024-03-21T08:47:09.842 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kZSTD supported: 0 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kXpressCompression supported: 0 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kLZ4HCCompression supported: 1 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kZlibCompression supported: 1 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kSnappyCompression supported: 1 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kLZ4Compression supported: 1 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: kBZip2Compression supported: 0 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Fast CRC32 supported: Supported on x86 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: DMutex implementation: pthread_mutex_t 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-21T08:47:09.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.merge_operator: 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_filter: None 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_filter_factory: None 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.sst_partitioner_factory: None 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.memtable_factory: SkipListFactory 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.table_factory: BlockBasedTable 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c9722f1a20) 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: cache_index_and_filter_blocks: 1 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: pin_top_level_index_and_filter: 1 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_type: 0 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: data_block_index_type: 0 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_shortening: 1 2024-03-21T08:47:09.844 INFO:journalctl@ceph.mon.a.smithi087.stdout: data_block_hash_table_util_ratio: 0.750000 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: checksum: 4 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: no_block_cache: 0 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache: 0x55c9722f7090 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_name: BinnedLRUCache 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_options: 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: capacity : 536870912 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: num_shard_bits : 4 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: strict_capacity_limit : 0 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: high_pri_pool_ratio: 0.000 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_cache_compressed: (nil) 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: persistent_cache: (nil) 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_size: 4096 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_size_deviation: 10 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_restart_interval: 16 2024-03-21T08:47:09.845 INFO:journalctl@ceph.mon.a.smithi087.stdout: index_block_restart_interval: 1 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: metadata_block_size: 4096 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: partition_filters: 0 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: use_delta_encoding: 1 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: filter_policy: bloomfilter 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: whole_key_filtering: 1 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: verify_compression: 0 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: read_amp_bytes_per_bit: 0 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: format_version: 5 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: enable_index_compression: 1 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: block_align: 0 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: max_auto_readahead_size: 262144 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: prepopulate_block_cache: 0 2024-03-21T08:47:09.846 INFO:journalctl@ceph.mon.a.smithi087.stdout: initial_auto_readahead_size: 8192 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout: num_file_reads_for_auto_readahead: 2 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.write_buffer_size: 33554432 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_write_buffer_number: 2 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression: NoCompression 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression: Disabled 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.prefix_extractor: nullptr 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.num_levels: 7 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-21T08:47:09.847 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.window_bits: -14 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.level: 32767 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.strategy: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.enabled: false 2024-03-21T08:47:09.848 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.target_file_size_base: 67108864 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.target_file_size_multiplier: 1 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-21T08:47:09.849 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.arena_block_size: 1048576 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.disable_auto_compactions: 0 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-21T08:47:09.850 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.table_properties_collectors: 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.inplace_update_support: 0 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.inplace_update_num_locks: 10000 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.memtable_huge_page_size: 0 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.bloom_locality: 0 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.max_successive_merges: 0 2024-03-21T08:47:09.851 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.optimize_filters_for_hits: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.paranoid_file_checks: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.force_consistency_checks: 1 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.report_bg_io_stats: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.ttl: 2592000 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.periodic_compaction_seconds: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.enable_blob_files: false 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.min_blob_size: 0 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.blob_file_size: 268435456 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.blob_compression_type: NoCompression 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.enable_blob_garbage_collection: false 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-21T08:47:09.852 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.blob_file_starting_level: 0 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 9b6186c7-50a2-4d40-bfda-9b1d871eaebe 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711010829832811, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711010829833809, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 95395, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 316, "table_properties": {"data_size": 93381, "index_size": 296, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13801, "raw_average_key_size": 50, "raw_value_size": 85854, "raw_average_value_size": 312, "num_data_blocks": 11, "num_entries": 275, "num_filter_entries": 275, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1711010829, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "9b6186c7-50a2-4d40-bfda-9b1d871eaebe", "db_session_id": "JQXRSWCP7ZWUTO9R9P45", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: EVENT_LOG_v1 {"time_micros": 1711010829833889, "job": 1, "event": "recovery_finished"} 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c9723ea000 2024-03-21T08:47:09.853 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: DB pointer 0x55c9723d4000 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** DB Stats ** 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** Compaction Stats [default] ** 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:47:09.854 INFO:journalctl@ceph.mon.a.smithi087.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: L0 2/0 94.99 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: Sum 2/0 94.99 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 139.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** Compaction Stats [default] ** 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 139.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-21T08:47:09.855 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: AddFile(Keys): cumulative 0, interval 0 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Cumulative compaction: 0.00 GB write, 31.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Interval compaction: 0.00 GB write, 31.12 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Block cache BinnedLRUCache@0x55c9722f7090#2 capacity: 512.00 MB usage: 1.39 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout: ** File Read Latency Histogram By Level [default] ** 2024-03-21T08:47:09.856 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: starting mon.a rank 0 at public addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] at bind addrs [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:09.913 INFO:teuthology.orchestra.run.smithi087.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-03-21T08:47:10.151 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???) e1 preinit fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:10.151 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).mds e1 new map 2024-03-21T08:47:10.151 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).mds e1 print_map 2024-03-21T08:47:10.151 INFO:journalctl@ceph.mon.a.smithi087.stdout: e1 2024-03-21T08:47:10.151 INFO:journalctl@ceph.mon.a.smithi087.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:47:10.151 INFO:journalctl@ceph.mon.a.smithi087.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout: legacy client fscid: -1 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout: 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout: No filesystems configured 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mon.a is new leader, mons a in quorum (ranks 0) 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: monmap e1: 1 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: fsmap 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: osdmap e1: 0 total, 0 up, 0 in 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 ceph-mon[32385]: mgrmap e1: no daemons active 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 bash[32366]: b1c901b5acae9f9a9cb3758fd5d36c814c02e635c57c0dac22dc45dd8623a752 2024-03-21T08:47:10.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:09 smithi087 systemd[1]: Started Ceph mon.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.354+0000 7f600b494700 1 Processor -- start 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.354+0000 7f600b494700 1 -- start start 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.354+0000 7f600b494700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600407c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.354+0000 7f600b494700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600407c8f0 con 0x7f600407dec0 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f6009230700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600407c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f6009230700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600407c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45288/0 (socket says 172.21.15.87:45288) 2024-03-21T08:47:11.028 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f6009230700 1 -- 172.21.15.87:0/2779933512 learned_addr learned my addr 172.21.15.87:0/2779933512 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f6009230700 1 -- 172.21.15.87:0/2779933512 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600407ca70 con 0x7f600407dec0 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f6009230700 1 --2- 172.21.15.87:0/2779933512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600407c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f60000098a0 tx=0x7f6000009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1ea6582dfcc740 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f5ffbfff700 1 -- 172.21.15.87:0/2779933512 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6000004160 con 0x7f600407dec0 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f5ffbfff700 1 -- 172.21.15.87:0/2779933512 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6000004300 con 0x7f600407dec0 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.355+0000 7f5ffbfff700 1 -- 172.21.15.87:0/2779933512 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6000011b50 con 0x7f600407dec0 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 -- 172.21.15.87:0/2779933512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 msgr2=0x7f600407c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 --2- 172.21.15.87:0/2779933512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600407c320 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f60000098a0 tx=0x7f6000009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 -- 172.21.15.87:0/2779933512 shutdown_connections 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 --2- 172.21.15.87:0/2779933512 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600407c320 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 -- 172.21.15.87:0/2779933512 >> 172.21.15.87:0/2779933512 conn(0x7f6004103070 msgr2=0x7f6004105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 -- 172.21.15.87:0/2779933512 shutdown_connections 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.356+0000 7f600b494700 1 -- 172.21.15.87:0/2779933512 wait complete. 2024-03-21T08:47:11.029 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f600b494700 1 Processor -- start 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f600b494700 1 -- start start 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f600b494700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600419dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f600b494700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600410e0c0 con 0x7f600407dec0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f6009230700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600419dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f6009230700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600419dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45302/0 (socket says 172.21.15.87:45302) 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f6009230700 1 -- 172.21.15.87:0/3230501795 learned_addr learned my addr 172.21.15.87:0/3230501795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f6009230700 1 -- 172.21.15.87:0/3230501795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600419e1a0 con 0x7f600407dec0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f6009230700 1 --2- 172.21.15.87:0/3230501795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600419dc60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6000011670 tx=0x7f6000011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.357+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f60000039c0 con 0x7f600407dec0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6000011e50 con 0x7f600407dec0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600419e430 con 0x7f600407dec0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f600001a5c0 con 0x7f600407dec0 2024-03-21T08:47:11.030 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60041a1120 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f600001ac50 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6000019a90 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.358+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60040637c0 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.361+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f6000003b60 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.430+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f60041a12f0 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f5ffa7fc700 1 -- 172.21.15.87:0/3230501795 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6000019c90 con 0x7f600407dec0 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 msgr2=0x7f600419dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f600b494700 1 --2- 172.21.15.87:0/3230501795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600419dc60 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6000011670 tx=0x7f6000011b60 comp rx=0 tx=0).stop 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 shutdown_connections 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f600b494700 1 --2- 172.21.15.87:0/3230501795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f600407dec0 0x7f600419dc60 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 >> 172.21.15.87:0/3230501795 conn(0x7f6004103070 msgr2=0x7f6004103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.432+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 shutdown_connections 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:10.433+0000 7f600b494700 1 -- 172.21.15.87:0/3230501795 wait complete. 2024-03-21T08:47:11.031 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-21T08:47:11.054 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:47:11.054 INFO:teuthology.orchestra.run.smithi087.stdout:Creating mgr... 2024-03-21T08:47:11.054 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-21T08:47:11.055 INFO:teuthology.orchestra.run.smithi087.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-21T08:47:11.301 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x 2024-03-21T08:47:11.301 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Failed to reset failed state of unit ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service: Unit ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service not loaded. 2024-03-21T08:47:11.442 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715.target.wants/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@.service. 2024-03-21T08:47:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:11 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3230501795' entity='client.admin' 2024-03-21T08:47:11.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:11 smithi087 systemd[1]: Starting Ceph mgr.x for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:47:11.944 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:11 smithi087 podman[32603]: 2024-03-21 08:47:11.577504958 +0000 UTC m=+0.016690236 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:47:11.944 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:11 smithi087 podman[32603]: 2024-03-21 08:47:11.8502447 +0000 UTC m=+0.289429979 container create 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T08:47:12.130 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:47:12.130 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-03-21T08:47:12.137 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:47:12.137 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-03-21T08:47:12.137 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-03-21T08:47:12.137 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to enable service . firewalld.service is not available 2024-03-21T08:47:12.145 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:47:12.145 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-03-21T08:47:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:47:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-03-21T08:47:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-03-21T08:47:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-21T08:47:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr to start... 2024-03-21T08:47:12.152 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr... 2024-03-21T08:47:12.276 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:12 smithi087 podman[32603]: 2024-03-21 08:47:12.044360022 +0000 UTC m=+0.483545313 container init 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, ceph=True) 2024-03-21T08:47:12.276 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:12 smithi087 podman[32603]: 2024-03-21 08:47:12.046914619 +0000 UTC m=+0.486099904 container start 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:47:12.276 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:12 smithi087 bash[32603]: 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 2024-03-21T08:47:12.277 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:12 smithi087 systemd[1]: Started Ceph mgr.x for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:47:12.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:12 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2398880425' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:47:12.909 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:12.601+0000 7f66307da200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:47:12.909 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:12.699+0000 7f66307da200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:47:13.262 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:13.263 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:47:13.264 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:47:13.265 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:47:13.273 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:47:13.274 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.275 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:47:04.721038+0000", 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.588+0000 7fb894fb3700 1 Processor -- start 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.588+0000 7fb894fb3700 1 -- start start 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb894fb3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89007c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb894fb3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89007c8d0 con 0x7fb89007dea0 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb88e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89007c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb88e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89007c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45314/0 (socket says 172.21.15.87:45314) 2024-03-21T08:47:13.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb88e59c700 1 -- 172.21.15.87:0/715971939 learned_addr learned my addr 172.21.15.87:0/715971939 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb88e59c700 1 -- 172.21.15.87:0/715971939 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89007ca50 con 0x7fb89007dea0 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.589+0000 7fb88e59c700 1 --2- 172.21.15.87:0/715971939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89007c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8800098a0 tx=0x7fb880009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9372eca66e884ec server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb88d59a700 1 -- 172.21.15.87:0/715971939 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb880004160 con 0x7fb89007dea0 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb88d59a700 1 -- 172.21.15.87:0/715971939 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb880004300 con 0x7fb89007dea0 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb88d59a700 1 -- 172.21.15.87:0/715971939 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb880011b50 con 0x7fb89007dea0 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb894fb3700 1 -- 172.21.15.87:0/715971939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 msgr2=0x7fb89007c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb894fb3700 1 --2- 172.21.15.87:0/715971939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89007c300 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb8800098a0 tx=0x7fb880009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb894fb3700 1 -- 172.21.15.87:0/715971939 shutdown_connections 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb894fb3700 1 --2- 172.21.15.87:0/715971939 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89007c300 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.590+0000 7fb894fb3700 1 -- 172.21.15.87:0/715971939 >> 172.21.15.87:0/715971939 conn(0x7fb890103070 msgr2=0x7fb890105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.591+0000 7fb894fb3700 1 -- 172.21.15.87:0/715971939 shutdown_connections 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.591+0000 7fb894fb3700 1 -- 172.21.15.87:0/715971939 wait complete. 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.591+0000 7fb894fb3700 1 Processor -- start 2024-03-21T08:47:13.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.591+0000 7fb894fb3700 1 -- start start 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.591+0000 7fb894fb3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.591+0000 7fb894fb3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89010e0c0 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb88e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb88e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45316/0 (socket says 172.21.15.87:45316) 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb88e59c700 1 -- 172.21.15.87:0/2398880425 learned_addr learned my addr 172.21.15.87:0/2398880425 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb88e59c700 1 -- 172.21.15.87:0/2398880425 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb89019e1c0 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb88e59c700 1 --2- 172.21.15.87:0/2398880425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89019dc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb880009fb0 tx=0x7fb880011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb8800038b0 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89019e450 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb880011e50 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb880019420 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.592+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8901a1140 con 0x7fb89007dea0 2024-03-21T08:47:13.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.593+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fb880019a80 con 0x7fb89007dea0 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.593+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb880024730 con 0x7fb89007dea0 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.593+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb890069250 con 0x7fb89007dea0 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.597+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fb880043dd0 con 0x7fb89007dea0 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.669+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb8901a1700 con 0x7fb89007dea0 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.670+0000 7fb87f7fe700 1 -- 172.21.15.87:0/2398880425 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fb880032030 con 0x7fb89007dea0 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 msgr2=0x7fb89019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 --2- 172.21.15.87:0/2398880425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89019dc80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb880009fb0 tx=0x7fb880011bf0 comp rx=0 tx=0).stop 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 shutdown_connections 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 --2- 172.21.15.87:0/2398880425 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb89007dea0 0x7fb89019dc80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 >> 172.21.15.87:0/2398880425 conn(0x7fb890103070 msgr2=0x7fb890103c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 shutdown_connections 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:12.671+0000 7fb894fb3700 1 -- 172.21.15.87:0/2398880425 wait complete. 2024-03-21T08:47:13.279 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (1/15)... 2024-03-21T08:47:13.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:13.344+0000 7f66307da200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:47:13.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:13.436+0000 7f66307da200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:47:13.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:13.576+0000 7f66307da200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:47:14.010 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:13.671+0000 7f66307da200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:47:14.010 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:13.753+0000 7f66307da200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:47:14.010 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:14.011+0000 7f66307da200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:47:14.299 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:14.121+0000 7f66307da200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:47:14.300 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:14.205+0000 7f66307da200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:47:14.300 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:14.300+0000 7f66307da200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:47:14.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:14.477+0000 7f66307da200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:47:15.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:15 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:15.457+0000 7f66307da200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:47:15.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:15 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:15.591+0000 7f66307da200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:47:16.024 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:15 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3557056149' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:47:16.025 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:15 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:15.935+0000 7f66307da200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:47:16.025 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:16 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:16.025+0000 7f66307da200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:47:16.403 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:16 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:16.224+0000 7f66307da200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:47:16.436 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:16.436 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:16.436 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:47:16.436 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:47:16.436 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:47:16.437 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:47:16.438 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:47:16.447 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:47:16.447 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:47:16.448 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:16.449 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:47:04.721038+0000", 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a82a5700 1 Processor -- start 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a82a5700 1 -- start start 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a82a5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a0107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a82a5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a0108480 con 0x7f68a0107a90 2024-03-21T08:47:16.450 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a6041700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a0107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a6041700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a0107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45326/0 (socket says 172.21.15.87:45326) 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.702+0000 7f68a6041700 1 -- 172.21.15.87:0/3347751135 learned_addr learned my addr 172.21.15.87:0/3347751135 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.703+0000 7f68a6041700 1 -- 172.21.15.87:0/3347751135 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a0108c90 con 0x7f68a0107a90 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.703+0000 7f68a6041700 1 --2- 172.21.15.87:0/3347751135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a0107eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f689c0098a0 tx=0x7f689c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3f736776692959 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.703+0000 7f68a503f700 1 -- 172.21.15.87:0/3347751135 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c004160 con 0x7f68a0107a90 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.703+0000 7f68a503f700 1 -- 172.21.15.87:0/3347751135 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f689c004300 con 0x7f68a0107a90 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.703+0000 7f68a503f700 1 -- 172.21.15.87:0/3347751135 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c011b50 con 0x7f68a0107a90 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 -- 172.21.15.87:0/3347751135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 msgr2=0x7f68a0107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 --2- 172.21.15.87:0/3347751135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a0107eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f689c0098a0 tx=0x7f689c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 -- 172.21.15.87:0/3347751135 shutdown_connections 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 --2- 172.21.15.87:0/3347751135 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a0107eb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 -- 172.21.15.87:0/3347751135 >> 172.21.15.87:0/3347751135 conn(0x7f68a0103070 msgr2=0x7f68a0105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 -- 172.21.15.87:0/3347751135 shutdown_connections 2024-03-21T08:47:16.451 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 -- 172.21.15.87:0/3347751135 wait complete. 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.704+0000 7f68a82a5700 1 Processor -- start 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a82a5700 1 -- start start 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a82a5700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a019bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a82a5700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a010c040 con 0x7f68a0107a90 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a6041700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a019bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a6041700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a019bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45338/0 (socket says 172.21.15.87:45338) 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a6041700 1 -- 172.21.15.87:0/3557056149 learned_addr learned my addr 172.21.15.87:0/3557056149 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a6041700 1 -- 172.21.15.87:0/3557056149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a019c020 con 0x7f68a0107a90 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a6041700 1 --2- 172.21.15.87:0/3557056149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a019bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f689c009fb0 tx=0x7f689c011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c0038b0 con 0x7f68a0107a90 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a019c2b0 con 0x7f68a0107a90 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.705+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f689c011e50 con 0x7f68a0107a90 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.706+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f689c019550 con 0x7f68a0107a90 2024-03-21T08:47:16.452 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.706+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a019efa0 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.706+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a0069250 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.706+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f689c019850 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.706+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f689c02dcd0 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.708+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f689c02d440 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.782+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f68a019f170 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.782+0000 7f68977fe700 1 -- 172.21.15.87:0/3557056149 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f689c003a50 con 0x7f68a0107a90 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 msgr2=0x7f68a019bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 --2- 172.21.15.87:0/3557056149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a019bae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f689c009fb0 tx=0x7f689c011bf0 comp rx=0 tx=0).stop 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 shutdown_connections 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 --2- 172.21.15.87:0/3557056149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f68a0107a90 0x7f68a019bae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 >> 172.21.15.87:0/3557056149 conn(0x7f68a0103070 msgr2=0x7f68a0103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 shutdown_connections 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:15.783+0000 7f68a82a5700 1 -- 172.21.15.87:0/3557056149 wait complete. 2024-03-21T08:47:16.453 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (2/15)... 2024-03-21T08:47:16.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:16 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:16.411+0000 7f66307da200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:47:16.949 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:16 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:16.697+0000 7f66307da200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:47:16.949 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:16 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:16.950+0000 7f66307da200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:47:17.409 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:17 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:17.185+0000 7f66307da200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:47:18.310 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:17 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:17.984+0000 7f66307da200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:47:18.565 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:18.557+0000 7f66307da200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:47:18.859 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:18.661+0000 7f66307da200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:47:19.162 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:18.893+0000 7f66307da200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:47:19.162 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: Activating manager daemon x 2024-03-21T08:47:19.162 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: mgrmap e2: x(active, starting, since 0.0042699s) 2024-03-21T08:47:19.162 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:47:19.162 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:47:19.162 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: Manager daemon x is now available 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:47:19.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:18 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:47:19.560 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:19.561 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:47:19.562 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:47:19.563 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:47:04.721038+0000", 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f15ed2700 1 Processor -- start 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f15ed2700 1 -- start start 2024-03-21T08:47:19.564 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f15ed2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10106d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f15ed2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f10107350 con 0x7f3f10106960 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f0f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f0f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46926/0 (socket says 172.21.15.87:46926) 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.885+0000 7f3f0f7fe700 1 -- 172.21.15.87:0/1957589071 learned_addr learned my addr 172.21.15.87:0/1957589071 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.886+0000 7f3f0f7fe700 1 -- 172.21.15.87:0/1957589071 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f10107b60 con 0x7f3f10106960 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.886+0000 7f3f0f7fe700 1 --2- 172.21.15.87:0/1957589071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10106d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f000098a0 tx=0x7f3f00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63844ab4f679e5a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.886+0000 7f3f0e7fc700 1 -- 172.21.15.87:0/1957589071 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f00004160 con 0x7f3f10106960 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.886+0000 7f3f0e7fc700 1 -- 172.21.15.87:0/1957589071 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f00004300 con 0x7f3f10106960 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.886+0000 7f3f0e7fc700 1 -- 172.21.15.87:0/1957589071 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f0000fbc0 con 0x7f3f10106960 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 -- 172.21.15.87:0/1957589071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 msgr2=0x7f3f10106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 --2- 172.21.15.87:0/1957589071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10106d80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f3f000098a0 tx=0x7f3f00009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 -- 172.21.15.87:0/1957589071 shutdown_connections 2024-03-21T08:47:19.565 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 --2- 172.21.15.87:0/1957589071 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10106d80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 -- 172.21.15.87:0/1957589071 >> 172.21.15.87:0/1957589071 conn(0x7f3f10101f20 msgr2=0x7f3f10104340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 -- 172.21.15.87:0/1957589071 shutdown_connections 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.887+0000 7f3f15ed2700 1 -- 172.21.15.87:0/1957589071 wait complete. 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f15ed2700 1 Processor -- start 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f15ed2700 1 -- start start 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f15ed2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f15ed2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1007aff0 con 0x7f3f10106960 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f0f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f0f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46942/0 (socket says 172.21.15.87:46942) 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f0f7fe700 1 -- 172.21.15.87:0/3117319067 learned_addr learned my addr 172.21.15.87:0/3117319067 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f0f7fe700 1 -- 172.21.15.87:0/3117319067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f10199dc0 con 0x7f3f10106960 2024-03-21T08:47:19.566 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.888+0000 7f3f0f7fe700 1 --2- 172.21.15.87:0/3117319067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10199880 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000f8c0 tx=0x7f3f0000ffb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.889+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f00003ad0 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.889+0000 7f3f15ed2700 1 -- 172.21.15.87:0/3117319067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1019a050 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.889+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3f0001d070 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.889+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3f00019560 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.889+0000 7f3f15ed2700 1 -- 172.21.15.87:0/3117319067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1019cd40 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.890+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3f00028060 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.890+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3f0002a790 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.890+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10069250 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.892+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3f00015070 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.908+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 2) v1 ==== 46050+0+0 (secure 0 0 0) 0x7f3f0001f030 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.975+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3f10062da0 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.975+0000 7f3f0cff9700 1 -- 172.21.15.87:0/3117319067 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f3f000207c0 con 0x7f3f10106960 2024-03-21T08:47:19.567 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.976+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 msgr2=0x7f3f10199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.976+0000 7f3ef67fc700 1 --2- 172.21.15.87:0/3117319067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10199880 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3f0000f8c0 tx=0x7f3f0000ffb0 comp rx=0 tx=0).stop 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.976+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 shutdown_connections 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.976+0000 7f3ef67fc700 1 --2- 172.21.15.87:0/3117319067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f10106960 0x7f3f10199880 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.976+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 >> 172.21.15.87:0/3117319067 conn(0x7f3f10101f20 msgr2=0x7f3f1018fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.976+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 shutdown_connections 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:18.977+0000 7f3ef67fc700 1 -- 172.21.15.87:0/3117319067 wait complete. 2024-03-21T08:47:19.568 INFO:teuthology.orchestra.run.smithi087.stdout:mgr not available, waiting (3/15)... 2024-03-21T08:47:20.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:19 smithi087 ceph-mon[32385]: from='mgr.14100 172.21.15.87:0/3999533920' entity='mgr.x' 2024-03-21T08:47:20.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:19 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3117319067' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:47:20.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:19 smithi087 ceph-mon[32385]: mgrmap e3: x(active, since 1.00646s) 2024-03-21T08:47:22.270 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:21 smithi087 ceph-mon[32385]: mgrmap e4: x(active, since 2s) 2024-03-21T08:47:22.875 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:22.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:22.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:47:22.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "health": { 2024-03-21T08:47:22.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-21T08:47:22.876 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 0 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:22.877 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "a" 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-21T08:47:22.878 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-21T08:47:22.879 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-21T08:47:22.880 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-21T08:47:22.881 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.881 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-21T08:47:22.881 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:22.881 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-21T08:47:22.881 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-21T08:47:22.881 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "restful" 2024-03-21T08:47:22.882 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ], 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "modified": "2024-03-21T08:47:04.721038+0000", 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout }, 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8ac0d57700 1 Processor -- start 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8ac0d57700 1 -- start start 2024-03-21T08:47:22.883 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8ac0d57700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc1067f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8ac0d57700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc07bf50 con 0x7f8abc1063d0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc1067f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc1067f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46984/0 (socket says 172.21.15.87:46984) 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.988+0000 7f8aba59c700 1 -- 172.21.15.87:0/1444883999 learned_addr learned my addr 172.21.15.87:0/1444883999 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.989+0000 7f8aba59c700 1 -- 172.21.15.87:0/1444883999 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc106d30 con 0x7f8abc1063d0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.989+0000 7f8aba59c700 1 --2- 172.21.15.87:0/1444883999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc1067f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0098a0 tx=0x7f8aac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72382b8bc287bce server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.989+0000 7f8ab959a700 1 -- 172.21.15.87:0/1444883999 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8aac004160 con 0x7f8abc1063d0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.989+0000 7f8ab959a700 1 -- 172.21.15.87:0/1444883999 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8aac004300 con 0x7f8abc1063d0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.989+0000 7f8ab959a700 1 -- 172.21.15.87:0/1444883999 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8aac011b50 con 0x7f8abc1063d0 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 -- 172.21.15.87:0/1444883999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 msgr2=0x7f8abc1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 --2- 172.21.15.87:0/1444883999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc1067f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8aac0098a0 tx=0x7f8aac009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 -- 172.21.15.87:0/1444883999 shutdown_connections 2024-03-21T08:47:22.884 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 --2- 172.21.15.87:0/1444883999 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc1067f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 -- 172.21.15.87:0/1444883999 >> 172.21.15.87:0/1444883999 conn(0x7f8abc101f90 msgr2=0x7f8abc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 -- 172.21.15.87:0/1444883999 shutdown_connections 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 -- 172.21.15.87:0/1444883999 wait complete. 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 Processor -- start 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.990+0000 7f8ac0d57700 1 -- start start 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8ac0d57700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc19dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8ac0d57700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc107dc0 con 0x7f8abc1063d0 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc19dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aba59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc19dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46992/0 (socket says 172.21.15.87:46992) 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aba59c700 1 -- 172.21.15.87:0/664873427 learned_addr learned my addr 172.21.15.87:0/664873427 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aba59c700 1 -- 172.21.15.87:0/664873427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc19e2d0 con 0x7f8abc1063d0 2024-03-21T08:47:22.885 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aba59c700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc19dd90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8aac011b60 tx=0x7f8aac011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8aac003970 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc19e560 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8aac003b10 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8aac024610 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.991+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc1a1210 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.992+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7f8aac024910 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.992+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc069250 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.992+0000 7f8aab7fe700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f8aa4039130 0x7f8aa403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.992+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f8aac011e00 con 0x7f8abc1063d0 2024-03-21T08:47:22.886 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.993+0000 7f8ab9d9b700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f8aa4039130 0x7f8aa403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.993+0000 7f8ab9d9b700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f8aa4039130 0x7f8aa403b5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:21.998+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8aac00b9e0 con 0x7f8abc1063d0 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.272+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f8abc02dde0 con 0x7f8abc1063d0 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.273+0000 7f8aab7fe700 1 -- 172.21.15.87:0/664873427 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f8aac016db0 con 0x7f8abc1063d0 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f8aa4039130 msgr2=0x7f8aa403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f8aa4039130 0x7f8aa403b5f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0006fd0 tx=0x7f8ab0006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:22.887 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 msgr2=0x7f8abc19dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc19dd90 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8aac011b60 tx=0x7f8aac011c40 comp rx=0 tx=0).stop 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 shutdown_connections 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f8aa4039130 0x7f8aa403b5f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 --2- 172.21.15.87:0/664873427 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8abc1063d0 0x7f8abc19dd90 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 >> 172.21.15.87:0/664873427 conn(0x7f8abc101f90 msgr2=0x7f8abc194980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 shutdown_connections 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:22.276+0000 7f8ac0d57700 1 -- 172.21.15.87:0/664873427 wait complete. 2024-03-21T08:47:22.888 INFO:teuthology.orchestra.run.smithi087.stdout:mgr is available 2024-03-21T08:47:23.262 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:22 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/664873427' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-21T08:47:24.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:23 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1197678716' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-21T08:47:24.067 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:24.067 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [global] 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout fsid = 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout [osd] 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-03-21T08:47:24.068 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72f517700 1 Processor -- start 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72f517700 1 -- start start 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72f517700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd728107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72f517700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd728108540 con 0x7fd728107b50 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72d2b3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd728107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72d2b3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd728107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:46996/0 (socket says 172.21.15.87:46996) 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.282+0000 7fd72d2b3700 1 -- 172.21.15.87:0/4139051654 learned_addr learned my addr 172.21.15.87:0/4139051654 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd72d2b3700 1 -- 172.21.15.87:0/4139051654 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd728108d00 con 0x7fd728107b50 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd72d2b3700 1 --2- 172.21.15.87:0/4139051654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd728107f70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098a0 tx=0x7fd724009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840ee65b4c317a4e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd71ffff700 1 -- 172.21.15.87:0/4139051654 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd724004160 con 0x7fd728107b50 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd71ffff700 1 -- 172.21.15.87:0/4139051654 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd724004300 con 0x7fd728107b50 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd71ffff700 1 -- 172.21.15.87:0/4139051654 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd724011b50 con 0x7fd728107b50 2024-03-21T08:47:24.069 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd72f517700 1 -- 172.21.15.87:0/4139051654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 msgr2=0x7fd728107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.283+0000 7fd72f517700 1 --2- 172.21.15.87:0/4139051654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd728107f70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7240098a0 tx=0x7fd724009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 -- 172.21.15.87:0/4139051654 shutdown_connections 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 --2- 172.21.15.87:0/4139051654 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd728107f70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 -- 172.21.15.87:0/4139051654 >> 172.21.15.87:0/4139051654 conn(0x7fd728103130 msgr2=0x7fd728105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 -- 172.21.15.87:0/4139051654 shutdown_connections 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 -- 172.21.15.87:0/4139051654 wait complete. 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 Processor -- start 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 -- start start 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd72819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.284+0000 7fd72f517700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd72810c0b0 con 0x7fd728107b50 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72d2b3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd72819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:24.070 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72d2b3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd72819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47002/0 (socket says 172.21.15.87:47002) 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72d2b3700 1 -- 172.21.15.87:0/1197678716 learned_addr learned my addr 172.21.15.87:0/1197678716 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72d2b3700 1 -- 172.21.15.87:0/1197678716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd72819c060 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72d2b3700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd72819bb20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd724009fb0 tx=0x7fd724011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd7240038b0 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd724011e50 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd72819c2f0 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd724019550 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.285+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd72819efe0 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.286+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7fd724019850 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.286+0000 7fd71e7fc700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fd7140390e0 0x7fd71403b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.286+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd728069250 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.286+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd72404d610 con 0x7fd728107b50 2024-03-21T08:47:24.071 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.286+0000 7fd72cab2700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fd7140390e0 0x7fd71403b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.287+0000 7fd72cab2700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fd7140390e0 0x7fd71403b5a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.291+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd72404d030 con 0x7fd728107b50 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.482+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd72819f1b0 con 0x7fd728107b50 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.483+0000 7fd71e7fc700 1 -- 172.21.15.87:0/1197678716 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd72819f1b0 con 0x7fd728107b50 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fd7140390e0 msgr2=0x7fd71403b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fd7140390e0 0x7fd71403b5a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd718006fd0 tx=0x7fd718006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 msgr2=0x7fd72819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd72819bb20 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd724009fb0 tx=0x7fd724011bf0 comp rx=0 tx=0).stop 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 shutdown_connections 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fd7140390e0 0x7fd71403b5a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 --2- 172.21.15.87:0/1197678716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd728107b50 0x7fd72819bb20 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 >> 172.21.15.87:0/1197678716 conn(0x7fd728103130 msgr2=0x7fd728103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 shutdown_connections 2024-03-21T08:47:24.072 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:23.485+0000 7fd72f517700 1 -- 172.21.15.87:0/1197678716 wait complete. 2024-03-21T08:47:24.073 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling cephadm module... 2024-03-21T08:47:25.012 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:24 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3673877803' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-21T08:47:25.409 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: ignoring --setuser ceph since I am not root 2024-03-21T08:47:25.409 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: ignoring --setgroup ceph since I am not root 2024-03-21T08:47:25.706 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:25.522+0000 7f6e9d128200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:47:25.706 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:25.615+0000 7f6e9d128200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:47:25.714 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc687005700 1 Processor -- start 2024-03-21T08:47:25.714 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc687005700 1 -- start start 2024-03-21T08:47:25.714 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc687005700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:25.714 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc687005700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68007c8f0 con 0x7fc68007dec0 2024-03-21T08:47:25.714 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc684da1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:25.714 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc684da1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47010/0 (socket says 172.21.15.87:47010) 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc684da1700 1 -- 172.21.15.87:0/2330936303 learned_addr learned my addr 172.21.15.87:0/2330936303 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc684da1700 1 -- 172.21.15.87:0/2330936303 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68007ca70 con 0x7fc68007dec0 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc684da1700 1 --2- 172.21.15.87:0/2330936303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68007c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc6740098a0 tx=0x7fc674009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9ea64708f2a5c18 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.501+0000 7fc67f7fe700 1 -- 172.21.15.87:0/2330936303 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc674004160 con 0x7fc68007dec0 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc67f7fe700 1 -- 172.21.15.87:0/2330936303 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc674004300 con 0x7fc68007dec0 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc67f7fe700 1 -- 172.21.15.87:0/2330936303 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc674011b50 con 0x7fc68007dec0 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 -- 172.21.15.87:0/2330936303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 msgr2=0x7fc68007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 --2- 172.21.15.87:0/2330936303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68007c320 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc6740098a0 tx=0x7fc674009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 -- 172.21.15.87:0/2330936303 shutdown_connections 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 --2- 172.21.15.87:0/2330936303 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68007c320 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:25.715 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 -- 172.21.15.87:0/2330936303 >> 172.21.15.87:0/2330936303 conn(0x7fc6801030c0 msgr2=0x7fc6801054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 -- 172.21.15.87:0/2330936303 shutdown_connections 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.502+0000 7fc687005700 1 -- 172.21.15.87:0/2330936303 wait complete. 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc687005700 1 Processor -- start 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc687005700 1 -- start start 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc687005700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc687005700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc68010e090 con 0x7fc68007dec0 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc684da1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc684da1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47018/0 (socket says 172.21.15.87:47018) 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc684da1700 1 -- 172.21.15.87:0/3673877803 learned_addr learned my addr 172.21.15.87:0/3673877803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc684da1700 1 -- 172.21.15.87:0/3673877803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68019e1c0 con 0x7fc68007dec0 2024-03-21T08:47:25.716 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc684da1700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68019dc80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc674011670 tx=0x7fc674011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.503+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc6740039c0 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.504+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc68019e450 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.504+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc674011e50 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.504+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc67401a5c0 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.504+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6801a1140 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.505+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 4) v1 ==== 46283+0+0 (secure 0 0 0) 0x7fc67401a8c0 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.505+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6800637c0 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.505+0000 7fc67dffb700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fc6700390e0 0x7fc67003b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.505+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc67404d6c0 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.505+0000 7fc67ffff700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fc6700390e0 0x7fc67003b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.505+0000 7fc67ffff700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fc6700390e0 0x7fc67003b5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006fd0 tx=0x7fc66c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.510+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc674019c70 con 0x7fc68007dec0 2024-03-21T08:47:25.717 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.761+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fc6801a1310 con 0x7fc68007dec0 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.969+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fc6801a1310 con 0x7fc68007dec0 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.970+0000 7fc67dffb700 1 -- 172.21.15.87:0/3673877803 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fc67402d910 con 0x7fc68007dec0 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.971+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fc6700390e0 msgr2=0x7fc67003b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.971+0000 7fc687005700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fc6700390e0 0x7fc67003b5a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc66c006fd0 tx=0x7fc66c006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.971+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 msgr2=0x7fc68019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.971+0000 7fc687005700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68019dc80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc674011670 tx=0x7fc674011b60 comp rx=0 tx=0).stop 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.971+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 shutdown_connections 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.971+0000 7fc687005700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fc6700390e0 0x7fc67003b5a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.972+0000 7fc687005700 1 --2- 172.21.15.87:0/3673877803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc68007dec0 0x7fc68019dc80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.972+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 >> 172.21.15.87:0/3673877803 conn(0x7fc6801030c0 msgr2=0x7fc680103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.972+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 shutdown_connections 2024-03-21T08:47:25.718 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:24.972+0000 7fc687005700 1 -- 172.21.15.87:0/3673877803 wait complete. 2024-03-21T08:47:26.268 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:25 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3673877803' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-21T08:47:26.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:25 smithi087 ceph-mon[32385]: mgrmap e5: x(active, since 6s) 2024-03-21T08:47:26.608 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:26.271+0000 7f6e9d128200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:47:26.608 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:26.365+0000 7f6e9d128200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:47:26.608 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:26.515+0000 7f6e9d128200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:47:26.608 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:26.610+0000 7f6e9d128200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:47:26.909 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:26.695+0000 7f6e9d128200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:47:26.998 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:27.004 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf1bbe8700 1 Processor -- start 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf1bbe8700 1 -- start start 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf1bbe8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf141083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf1bbe8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf14108990 con 0x7fdf14107fa0 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf19984700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf141083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf19984700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf141083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47052/0 (socket says 172.21.15.87:47052) 2024-03-21T08:47:27.005 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.128+0000 7fdf19984700 1 -- 172.21.15.87:0/20335216 learned_addr learned my addr 172.21.15.87:0/20335216 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.129+0000 7fdf19984700 1 -- 172.21.15.87:0/20335216 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14106c50 con 0x7fdf14107fa0 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.129+0000 7fdf19984700 1 --2- 172.21.15.87:0/20335216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf141083c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf100098a0 tx=0x7fdf10009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a07cb4ac348e381 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.129+0000 7fdf18982700 1 -- 172.21.15.87:0/20335216 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf10004160 con 0x7fdf14107fa0 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.129+0000 7fdf18982700 1 -- 172.21.15.87:0/20335216 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf10004300 con 0x7fdf14107fa0 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.129+0000 7fdf18982700 1 -- 172.21.15.87:0/20335216 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf10011b50 con 0x7fdf14107fa0 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/20335216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 msgr2=0x7fdf141083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 --2- 172.21.15.87:0/20335216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf141083c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fdf100098a0 tx=0x7fdf10009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/20335216 shutdown_connections 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 --2- 172.21.15.87:0/20335216 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf141083c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/20335216 >> 172.21.15.87:0/20335216 conn(0x7fdf141016e0 msgr2=0x7fdf14103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/20335216 shutdown_connections 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/20335216 wait complete. 2024-03-21T08:47:27.006 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 Processor -- start 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- start start 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf14191bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.130+0000 7fdf1bbe8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1410b010 con 0x7fdf14107fa0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf19984700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf14191bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf19984700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf14191bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47060/0 (socket says 172.21.15.87:47060) 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf19984700 1 -- 172.21.15.87:0/1416392138 learned_addr learned my addr 172.21.15.87:0/1416392138 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf19984700 1 -- 172.21.15.87:0/1416392138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf14192110 con 0x7fdf14107fa0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf19984700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf14191bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf10011850 tx=0x7fdf10011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf10003af0 con 0x7fdf14107fa0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdf10020070 con 0x7fdf14107fa0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf14190300 con 0x7fdf14107fa0 2024-03-21T08:47:27.007 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fdf100195a0 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.131+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf14190840 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.132+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7fdf100198d0 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.132+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf14069250 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.132+0000 7fdf0affd700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 0x7fdf0003b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.132+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fdf10011e00 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.132+0000 7fdf19183700 1 -- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 msgr2=0x7fdf0003b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.132+0000 7fdf19183700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 0x7fdf0003b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.137+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf100172b0 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.333+0000 7fdf19183700 1 -- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 msgr2=0x7fdf0003b6f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.333+0000 7fdf19183700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 0x7fdf0003b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.395+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fdf14195950 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.396+0000 7fdf0affd700 1 -- 172.21.15.87:0/1416392138 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7fdf10003c90 con 0x7fdf14107fa0 2024-03-21T08:47:27.008 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 msgr2=0x7fdf0003b6f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 0x7fdf0003b6f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 msgr2=0x7fdf14191bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf14191bd0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdf10011850 tx=0x7fdf10011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 shutdown_connections 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7fdf00039230 0x7fdf0003b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 --2- 172.21.15.87:0/1416392138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdf14107fa0 0x7fdf14191bd0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 >> 172.21.15.87:0/1416392138 conn(0x7fdf141016e0 msgr2=0x7fdf14102610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 shutdown_connections 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:26.398+0000 7fdf1bbe8700 1 -- 172.21.15.87:0/1416392138 wait complete. 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for the mgr to restart... 2024-03-21T08:47:27.009 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr epoch 5... 2024-03-21T08:47:27.211 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1416392138' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T08:47:27.212 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:26.958+0000 7f6e9d128200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:47:27.212 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:27 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:27.073+0000 7f6e9d128200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:47:27.212 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:27 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:27.159+0000 7f6e9d128200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:47:27.631 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:27 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:27.257+0000 7f6e9d128200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:47:27.632 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:27 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:27.438+0000 7f6e9d128200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:47:28.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:28 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:28.384+0000 7f6e9d128200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:47:28.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:28 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:28.513+0000 7f6e9d128200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:47:29.126 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:28 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:28.842+0000 7f6e9d128200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:47:29.126 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:28 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:28.931+0000 7f6e9d128200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:47:29.126 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:29 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:29.128+0000 7f6e9d128200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:47:29.408 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:29 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:29.313+0000 7f6e9d128200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:47:29.864 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:29 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:29.607+0000 7f6e9d128200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:47:29.864 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:29 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:29.866+0000 7f6e9d128200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:47:30.158 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:30 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:30.103+0000 7f6e9d128200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:47:31.159 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:30 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:30.899+0000 7f6e9d128200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:47:31.773 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:31.461+0000 7f6e9d128200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:47:31.774 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:31.561+0000 7f6e9d128200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:47:31.774 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:31.776+0000 7f6e9d128200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: Active manager daemon x restarted 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: Activating manager daemon x 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: osdmap e2: 0 total, 0 up, 0 in 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: mgrmap e6: x(active, starting, since 0.00352822s) 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: Manager daemon x is now available 2024-03-21T08:47:32.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:32.160 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:32.160 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:47:32.160 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:31 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:47:33.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:32 smithi087 ceph-mon[32385]: Found migration_current of "None". Setting to last migration. 2024-03-21T08:47:33.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:32 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:47:33.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:32 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:47:33.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:32 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:33.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:32 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:33.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:32 smithi087 ceph-mon[32385]: mgrmap e7: x(active, since 1.00591s) 2024-03-21T08:47:33.428 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bed6c700 1 Processor -- start 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bed6c700 1 -- start start 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bed6c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b8106190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bed6c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8106760 con 0x7f98b8105d70 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bcb08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b8106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bcb08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b8106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47068/0 (socket says 172.21.15.87:47068) 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.428+0000 7f98bcb08700 1 -- 172.21.15.87:0/2080636605 learned_addr learned my addr 172.21.15.87:0/2080636605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:33.429 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.429+0000 7f98bcb08700 1 -- 172.21.15.87:0/2080636605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b81068e0 con 0x7f98b8105d70 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.429+0000 7f98bcb08700 1 --2- 172.21.15.87:0/2080636605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b8106190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0098a0 tx=0x7f98ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da366136677a6d93 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.429+0000 7f98b77fe700 1 -- 172.21.15.87:0/2080636605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac004160 con 0x7f98b8105d70 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.429+0000 7f98b77fe700 1 -- 172.21.15.87:0/2080636605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98ac004300 con 0x7f98b8105d70 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.429+0000 7f98b77fe700 1 -- 172.21.15.87:0/2080636605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac011b50 con 0x7f98b8105d70 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 -- 172.21.15.87:0/2080636605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 msgr2=0x7f98b8106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2080636605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b8106190 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0098a0 tx=0x7f98ac009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 -- 172.21.15.87:0/2080636605 shutdown_connections 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2080636605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b8106190 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 -- 172.21.15.87:0/2080636605 >> 172.21.15.87:0/2080636605 conn(0x7f98b81016f0 msgr2=0x7f98b8103b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 -- 172.21.15.87:0/2080636605 shutdown_connections 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 -- 172.21.15.87:0/2080636605 wait complete. 2024-03-21T08:47:33.430 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 Processor -- start 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.430+0000 7f98bed6c700 1 -- start start 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bed6c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b819ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bed6c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8109c30 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bcb08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b819ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bcb08700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b819ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47078/0 (socket says 172.21.15.87:47078) 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bcb08700 1 -- 172.21.15.87:0/2104848664 learned_addr learned my addr 172.21.15.87:0/2104848664 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bcb08700 1 -- 172.21.15.87:0/2104848664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b819bfd0 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bcb08700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b819ba90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f98ac011850 tx=0x7f98ac011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac003af0 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f98ac020070 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98b819c2c0 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.431+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f98ac019550 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.432+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b819ef50 con 0x7f98b8105d70 2024-03-21T08:47:33.431 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.432+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 5) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f98ac019800 con 0x7f98b8105d70 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.433+0000 7f98b5ffb700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.433+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 --> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98a803bda0 con 0x7f98a80391d0 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.433+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f98ac011e00 con 0x7f98b8105d70 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.433+0000 7f98b7fff700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 msgr2=0x7f98a803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.433+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.634+0000 7f98b7fff700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 msgr2=0x7f98a803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:27.634+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:28.035+0000 7f98b7fff700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 msgr2=0x7f98a803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:28.035+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:28.836+0000 7f98b7fff700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 msgr2=0x7f98a803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:28.836+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:30.437+0000 7f98b7fff700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 msgr2=0x7f98a803b690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/277415650 2024-03-21T08:47:33.432 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:30.437+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:31.791+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mgrmap(e 6) v1 ==== 46061+0+0 (secure 0 0 0) 0x7f98ac00b3e0 con 0x7f98b8105d70 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:31.791+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 msgr2=0x7f98a803b690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:31.791+0000 7f98b5ffb700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.793+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7f98ac04ea30 con 0x7f98b8105d70 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.793+0000 7f98b5ffb700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f98a803cb10 0x7f98a803ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.793+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f98a803bda0 con 0x7f98a803cb10 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.793+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f98a803cb10 0x7f98a803ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.794+0000 7f98b7fff700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f98a803cb10 0x7f98a803ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f98a4003a10 tx=0x7f98a40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.794+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f98a803bda0 con 0x7f98a803cb10 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f98b8069250 con 0x7f98a803cb10 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98b5ffb700 1 -- 172.21.15.87:0/2104848664 <== mgr.14120 v2:172.21.15.87:6800/2393104803 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f98b8069250 con 0x7f98a803cb10 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f98a803cb10 msgr2=0x7f98a803ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f98a803cb10 0x7f98a803ef00 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f98a4003a10 tx=0x7f98a40092b0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.433 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 msgr2=0x7f98b819ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b819ba90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f98ac011850 tx=0x7f98ac011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 shutdown_connections 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f98a803cb10 0x7f98a803ef00 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f98b8105d70 0x7f98b819ba90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 --2- 172.21.15.87:0/2104848664 >> [v2:172.21.15.87:6800/277415650,v1:172.21.15.87:6801/277415650] conn(0x7f98a80391d0 0x7f98a803b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 >> 172.21.15.87:0/2104848664 conn(0x7f98b81016f0 msgr2=0x7f98b81022a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 shutdown_connections 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:32.801+0000 7f98bed6c700 1 -- 172.21.15.87:0/2104848664 wait complete. 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:mgr epoch 5 is available 2024-03-21T08:47:33.434 INFO:teuthology.orchestra.run.smithi087.stdout:Setting orchestrator backend to cephadm... 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:32] ENGINE Bus STARTING 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:32] ENGINE Serving on https://172.21.15.87:7150 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:32] ENGINE Serving on http://172.21.15.87:8765 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:32] ENGINE Bus STARTED 2024-03-21T08:47:34.047 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:33 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a871d700 1 Processor -- start 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a871d700 1 -- start start 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a871d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a010a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a871d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a007be90 con 0x7f56a0108530 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a64b9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a010a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a64b9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a010a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42938/0 (socket says 172.21.15.87:42938) 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.826+0000 7f56a64b9700 1 -- 172.21.15.87:0/73695328 learned_addr learned my addr 172.21.15.87:0/73695328 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a64b9700 1 -- 172.21.15.87:0/73695328 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a010ae60 con 0x7f56a0108530 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a64b9700 1 --2- 172.21.15.87:0/73695328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a010a920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f569c0098a0 tx=0x7f569c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f1fceabc5f529e4 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a54b7700 1 -- 172.21.15.87:0/73695328 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f569c004160 con 0x7f56a0108530 2024-03-21T08:47:34.667 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a54b7700 1 -- 172.21.15.87:0/73695328 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f569c004300 con 0x7f56a0108530 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a54b7700 1 -- 172.21.15.87:0/73695328 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f569c011b50 con 0x7f56a0108530 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a871d700 1 -- 172.21.15.87:0/73695328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 msgr2=0x7f56a010a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.827+0000 7f56a871d700 1 --2- 172.21.15.87:0/73695328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a010a920 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f569c0098a0 tx=0x7f569c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 -- 172.21.15.87:0/73695328 shutdown_connections 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 --2- 172.21.15.87:0/73695328 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a010a920 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 -- 172.21.15.87:0/73695328 >> 172.21.15.87:0/73695328 conn(0x7f56a0101f40 msgr2=0x7f56a0104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 -- 172.21.15.87:0/73695328 shutdown_connections 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 -- 172.21.15.87:0/73695328 wait complete. 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 Processor -- start 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 -- start start 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a019ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.828+0000 7f56a871d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a010be80 con 0x7f56a0108530 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a64b9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a019ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:34.668 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a64b9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a019ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42946/0 (socket says 172.21.15.87:42946) 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a64b9700 1 -- 172.21.15.87:0/2388303026 learned_addr learned my addr 172.21.15.87:0/2388303026 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a64b9700 1 -- 172.21.15.87:0/2388303026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a019bf50 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a64b9700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a019ba10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f569c011670 tx=0x7f569c011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f569c0039c0 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f569c011e50 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f569c01a5c0 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a019c1e0 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.829+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a019eed0 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.830+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 7) v1 ==== 46188+0+0 (secure 0 0 0) 0x7f569c024430 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.830+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56a00637c0 con 0x7f56a0108530 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.830+0000 7f56977fe700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f568c039070 0x7f568c03b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:34.669 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.830+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f569c04d3d0 con 0x7f56a0108530 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.830+0000 7f56a5cb8700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f568c039070 0x7f568c03b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.831+0000 7f56a5cb8700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f568c039070 0x7f568c03b530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5690006fd0 tx=0x7f5690006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.835+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f569c02d910 con 0x7f56a0108530 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:33.843+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f569c02d3f0 con 0x7f56a0108530 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.049+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f56a0103190 con 0x7f568c039070 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.056+0000 7f56977fe700 1 -- 172.21.15.87:0/2388303026 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f56a0103190 con 0x7f568c039070 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f568c039070 msgr2=0x7f568c03b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f568c039070 0x7f568c03b530 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5690006fd0 tx=0x7f5690006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 msgr2=0x7f56a019ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a019ba10 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f569c011670 tx=0x7f569c011b60 comp rx=0 tx=0).stop 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 shutdown_connections 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f568c039070 0x7f568c03b530 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:34.670 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 --2- 172.21.15.87:0/2388303026 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56a0108530 0x7f56a019ba10 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:34.671 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 >> 172.21.15.87:0/2388303026 conn(0x7f56a0101f40 msgr2=0x7f56a0102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:34.671 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 shutdown_connections 2024-03-21T08:47:34.671 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:34.058+0000 7f56a871d700 1 -- 172.21.15.87:0/2388303026 wait complete. 2024-03-21T08:47:34.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:34 smithi087 ceph-mon[32385]: mgrmap e8: x(active, since 2s) 2024-03-21T08:47:34.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:34 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:34.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:34 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.071+0000 7f3a19c59700 1 Processor -- start 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.071+0000 7f3a19c59700 1 -- start start 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.071+0000 7f3a19c59700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a14107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.071+0000 7f3a19c59700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14108540 con 0x7f3a14107b50 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a137fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a14107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a137fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a14107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42952/0 (socket says 172.21.15.87:42952) 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a137fe700 1 -- 172.21.15.87:0/2311601376 learned_addr learned my addr 172.21.15.87:0/2311601376 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a137fe700 1 -- 172.21.15.87:0/2311601376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14108d00 con 0x7f3a14107b50 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a137fe700 1 --2- 172.21.15.87:0/2311601376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a14107f70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a040098a0 tx=0x7f3a04009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb48e95e72b6b36 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:35.890 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a12ffd700 1 -- 172.21.15.87:0/2311601376 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a04004160 con 0x7f3a14107b50 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a12ffd700 1 -- 172.21.15.87:0/2311601376 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a04004300 con 0x7f3a14107b50 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.072+0000 7f3a12ffd700 1 -- 172.21.15.87:0/2311601376 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a04011b50 con 0x7f3a14107b50 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 -- 172.21.15.87:0/2311601376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 msgr2=0x7f3a14107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 --2- 172.21.15.87:0/2311601376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a14107f70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a040098a0 tx=0x7f3a04009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 -- 172.21.15.87:0/2311601376 shutdown_connections 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 --2- 172.21.15.87:0/2311601376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a14107f70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 -- 172.21.15.87:0/2311601376 >> 172.21.15.87:0/2311601376 conn(0x7f3a14103130 msgr2=0x7f3a14105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 -- 172.21.15.87:0/2311601376 shutdown_connections 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 -- 172.21.15.87:0/2311601376 wait complete. 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 Processor -- start 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.073+0000 7f3a19c59700 1 -- start start 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a19c59700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a1419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a19c59700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1410c0b0 con 0x7f3a14107b50 2024-03-21T08:47:35.891 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a137fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a1419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a137fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a1419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42962/0 (socket says 172.21.15.87:42962) 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a137fe700 1 -- 172.21.15.87:0/498530131 learned_addr learned my addr 172.21.15.87:0/498530131 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a137fe700 1 -- 172.21.15.87:0/498530131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a1419c060 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a137fe700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a1419bb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a04011850 tx=0x7f3a04011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a04003af0 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a04020070 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1419c2f0 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.074+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a040195a0 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.075+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1419efe0 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.075+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f3a04019850 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.075+0000 7f3a117fa700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f3a000391d0 0x7f3a0003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.075+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3a04011e00 con 0x7f3a14107b50 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.076+0000 7f3a0affd700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f3a000391d0 0x7f3a0003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:35.892 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.076+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14069250 con 0x7f3a14107b50 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.076+0000 7f3a0affd700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f3a000391d0 0x7f3a0003b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006fd0 tx=0x7f39fc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.081+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a04003c90 con 0x7f3a14107b50 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.276+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a14104390 con 0x7f3a000391d0 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.277+0000 7f3a117fa700 1 -- 172.21.15.87:0/498530131 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f3a14104390 con 0x7f3a000391d0 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f3a000391d0 msgr2=0x7f3a0003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f3a000391d0 0x7f3a0003b690 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f39fc006fd0 tx=0x7f39fc006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 msgr2=0x7f3a1419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a1419bb20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f3a04011850 tx=0x7f3a04011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 shutdown_connections 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f3a000391d0 0x7f3a0003b690 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 --2- 172.21.15.87:0/498530131 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a14107b50 0x7f3a1419bb20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 >> 172.21.15.87:0/498530131 conn(0x7f3a14103130 msgr2=0x7f3a14103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 shutdown_connections 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:35.279+0000 7f3a19c59700 1 -- 172.21.15.87:0/498530131 wait complete. 2024-03-21T08:47:35.893 INFO:teuthology.orchestra.run.smithi087.stdout:Generating ssh key... 2024-03-21T08:47:36.138 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:35 smithi087 ceph-mon[32385]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: Generating public/private rsa key pair. 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: Your identification has been saved in /tmp/tmptduzb7rz/key. 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: Your public key has been saved in /tmp/tmptduzb7rz/key.pub. 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: The key fingerprint is: 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: SHA256:vG8s5Tdmw/SV0DGDNwYtnG55H5d2Uz+T4c083/hCzl8 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: The key's randomart image is: 2024-03-21T08:47:36.846 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: +---[RSA 3072]----+ 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | ..= | 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | = X.| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | . Bo%| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | . = &B| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | S . +.&| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | .. . o.=| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | .+ o = oE| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | ..+ B = o| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: | o.+ o o.| 2024-03-21T08:47:36.847 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: +----[SHA256]-----+ 2024-03-21T08:47:37.141 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-mon[32385]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:37.142 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:37.142 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:36 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.371+0000 7f0f100fa700 1 Processor -- start 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f100fa700 1 -- start start 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f100fa700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f08107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f100fa700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f08108540 con 0x7f0f08107b50 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f0de96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f08107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f0de96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f08107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42966/0 (socket says 172.21.15.87:42966) 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f0de96700 1 -- 172.21.15.87:0/944560894 learned_addr learned my addr 172.21.15.87:0/944560894 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:37.247 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f0de96700 1 -- 172.21.15.87:0/944560894 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f08108d00 con 0x7f0f08107b50 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f0de96700 1 --2- 172.21.15.87:0/944560894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f08107f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f000098a0 tx=0x7f0f00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e40fff1a9c2e093b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.372+0000 7f0f0ce94700 1 -- 172.21.15.87:0/944560894 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f00004160 con 0x7f0f08107b50 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f0ce94700 1 -- 172.21.15.87:0/944560894 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f00004300 con 0x7f0f08107b50 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f0ce94700 1 -- 172.21.15.87:0/944560894 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f00011b50 con 0x7f0f08107b50 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 -- 172.21.15.87:0/944560894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 msgr2=0x7f0f08107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 --2- 172.21.15.87:0/944560894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f08107f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0f000098a0 tx=0x7f0f00009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 -- 172.21.15.87:0/944560894 shutdown_connections 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 --2- 172.21.15.87:0/944560894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f08107f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 -- 172.21.15.87:0/944560894 >> 172.21.15.87:0/944560894 conn(0x7f0f08103130 msgr2=0x7f0f08105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 -- 172.21.15.87:0/944560894 shutdown_connections 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.373+0000 7f0f100fa700 1 -- 172.21.15.87:0/944560894 wait complete. 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f100fa700 1 Processor -- start 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f100fa700 1 -- start start 2024-03-21T08:47:37.248 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f100fa700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f0819bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f100fa700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0810c0b0 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f0de96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f0819bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f0de96700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f0819bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:42968/0 (socket says 172.21.15.87:42968) 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f0de96700 1 -- 172.21.15.87:0/188217088 learned_addr learned my addr 172.21.15.87:0/188217088 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f0de96700 1 -- 172.21.15.87:0/188217088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0819c060 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.374+0000 7f0f0de96700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f0819bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f00011850 tx=0x7f0f00011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.375+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f00003af0 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.375+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f00020070 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.375+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f000195a0 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.375+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f0819c2f0 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.375+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0819efe0 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.375+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f0f00019850 con 0x7f0f08107b50 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.376+0000 7f0f06ffd700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0ef00391d0 0x7f0ef003b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:37.249 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.376+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f08069250 con 0x7f0f08107b50 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.376+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0f00011e00 con 0x7f0f08107b50 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.376+0000 7f0f0d695700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0ef00391d0 0x7f0ef003b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.376+0000 7f0f0d695700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0ef00391d0 0x7f0ef003b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0efc006fd0 tx=0x7f0efc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.381+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0f00018ba0 con 0x7f0f08107b50 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.577+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f08104390 con 0x7f0ef00391d0 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.732+0000 7f0f06ffd700 1 -- 172.21.15.87:0/188217088 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0f08104390 con 0x7f0ef00391d0 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0ef00391d0 msgr2=0x7f0ef003b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0ef00391d0 0x7f0ef003b690 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0efc006fd0 tx=0x7f0efc006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 msgr2=0x7f0f0819bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f0819bb20 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0f00011850 tx=0x7f0f00011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 shutdown_connections 2024-03-21T08:47:37.250 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0ef00391d0 0x7f0ef003b690 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:37.251 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 --2- 172.21.15.87:0/188217088 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0f08107b50 0x7f0f0819bb20 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:37.251 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 >> 172.21.15.87:0/188217088 conn(0x7f0f08103130 msgr2=0x7f0f08103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:37.251 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 shutdown_connections 2024-03-21T08:47:37.251 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:36.734+0000 7f0f100fa700 1 -- 172.21.15.87:0/188217088 wait complete. 2024-03-21T08:47:38.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:37 smithi087 ceph-mon[32385]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:38.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:37 smithi087 ceph-mon[32385]: Generating ssh key... 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf9fb8700 1 Processor -- start 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf9fb8700 1 -- start start 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf9fb8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf4107f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf9fb8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf4108540 con 0x7f6bf4107b50 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf4107f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf4107f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33106/0 (socket says 172.21.15.87:33106) 2024-03-21T08:47:38.479 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf37fe700 1 -- 172.21.15.87:0/1516786389 learned_addr learned my addr 172.21.15.87:0/1516786389 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf37fe700 1 -- 172.21.15.87:0/1516786389 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf4108d00 con 0x7f6bf4107b50 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.650+0000 7f6bf37fe700 1 --2- 172.21.15.87:0/1516786389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf4107f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6be40098a0 tx=0x7f6be4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8d1074d034fc89 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf27fc700 1 -- 172.21.15.87:0/1516786389 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be4004160 con 0x7f6bf4107b50 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf27fc700 1 -- 172.21.15.87:0/1516786389 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6be4004300 con 0x7f6bf4107b50 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf27fc700 1 -- 172.21.15.87:0/1516786389 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be4011b50 con 0x7f6bf4107b50 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/1516786389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 msgr2=0x7f6bf4107f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 --2- 172.21.15.87:0/1516786389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf4107f70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f6be40098a0 tx=0x7f6be4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/1516786389 shutdown_connections 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 --2- 172.21.15.87:0/1516786389 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf4107f70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/1516786389 >> 172.21.15.87:0/1516786389 conn(0x7f6bf4103130 msgr2=0x7f6bf4105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/1516786389 shutdown_connections 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.651+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/1516786389 wait complete. 2024-03-21T08:47:38.480 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf9fb8700 1 Processor -- start 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf9fb8700 1 -- start start 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf9fb8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf407db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf9fb8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf410c0b0 con 0x7f6bf4107b50 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf407db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf407db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33116/0 (socket says 172.21.15.87:33116) 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf37fe700 1 -- 172.21.15.87:0/775108687 learned_addr learned my addr 172.21.15.87:0/775108687 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.652+0000 7f6bf37fe700 1 -- 172.21.15.87:0/775108687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bf407e0d0 con 0x7f6bf4107b50 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.653+0000 7f6bf37fe700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf407db90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6be4011850 tx=0x7f6be4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.653+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be4003af0 con 0x7f6bf4107b50 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.653+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6be4020070 con 0x7f6bf4107b50 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.653+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf407c2c0 con 0x7f6bf4107b50 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.653+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6be40195a0 con 0x7f6bf4107b50 2024-03-21T08:47:38.481 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.653+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf407c800 con 0x7f6bf4107b50 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.654+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f6be40198a0 con 0x7f6bf4107b50 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.654+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf4069250 con 0x7f6bf4107b50 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.654+0000 7f6bf0ff9700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f6bdc039180 0x7f6bdc03b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.654+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6be4011e00 con 0x7f6bf4107b50 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.654+0000 7f6bf2ffd700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f6bdc039180 0x7f6bdc03b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.654+0000 7f6bf2ffd700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f6bdc039180 0x7f6bdc03b640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6be8006fd0 tx=0x7f6be8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.659+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6be4023530 con 0x7f6bf4107b50 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.855+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bf4104390 con 0x7f6bdc039180 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.855+0000 7f6bf0ff9700 1 -- 172.21.15.87:0/775108687 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f6bf4104390 con 0x7f6bdc039180 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f6bdc039180 msgr2=0x7f6bdc03b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f6bdc039180 0x7f6bdc03b640 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f6be8006fd0 tx=0x7f6be8006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 msgr2=0x7f6bf407db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:38.482 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf407db90 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6be4011850 tx=0x7f6be4011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 shutdown_connections 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f6bdc039180 0x7f6bdc03b640 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 --2- 172.21.15.87:0/775108687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6bf4107b50 0x7f6bf407db90 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 >> 172.21.15.87:0/775108687 conn(0x7f6bf4103130 msgr2=0x7f6bf4103c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 shutdown_connections 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:37.857+0000 7f6bf9fb8700 1 -- 172.21.15.87:0/775108687 wait complete. 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:Adding key to root@localhost authorized_keys... 2024-03-21T08:47:38.483 INFO:teuthology.orchestra.run.smithi087.stdout:Adding host smithi087... 2024-03-21T08:47:39.104 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:38 smithi087 ceph-mon[32385]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:40.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:40 smithi087 ceph-mon[32385]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi087", "addr": "172.21.15.87", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:40.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:40 smithi087 ceph-mon[32385]: Deploying cephadm binary to smithi087 2024-03-21T08:47:43.953 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Added host 'smithi087' with addr '172.21.15.87' 2024-03-21T08:47:43.953 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f274f1700 1 Processor -- start 2024-03-21T08:47:43.953 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f274f1700 1 -- start start 2024-03-21T08:47:43.953 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f274f1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2007c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:43.953 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f274f1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2007c8f0 con 0x7f9f2007dec0 2024-03-21T08:47:43.953 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f2528d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2007c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f2528d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2007c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33128/0 (socket says 172.21.15.87:33128) 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.897+0000 7f9f2528d700 1 -- 172.21.15.87:0/453073700 learned_addr learned my addr 172.21.15.87:0/453073700 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f2528d700 1 -- 172.21.15.87:0/453073700 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2007ca70 con 0x7f9f2007dec0 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f2528d700 1 --2- 172.21.15.87:0/453073700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2007c320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0098a0 tx=0x7f9f1c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bbeb16519e79c8c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f17fff700 1 -- 172.21.15.87:0/453073700 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f1c004160 con 0x7f9f2007dec0 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f17fff700 1 -- 172.21.15.87:0/453073700 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f1c004300 con 0x7f9f2007dec0 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f17fff700 1 -- 172.21.15.87:0/453073700 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f1c011b50 con 0x7f9f2007dec0 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f274f1700 1 -- 172.21.15.87:0/453073700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 msgr2=0x7f9f2007c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.898+0000 7f9f274f1700 1 --2- 172.21.15.87:0/453073700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2007c320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0098a0 tx=0x7f9f1c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 -- 172.21.15.87:0/453073700 shutdown_connections 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 --2- 172.21.15.87:0/453073700 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2007c320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 -- 172.21.15.87:0/453073700 >> 172.21.15.87:0/453073700 conn(0x7f9f201030c0 msgr2=0x7f9f201054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:43.954 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 -- 172.21.15.87:0/453073700 shutdown_connections 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 -- 172.21.15.87:0/453073700 wait complete. 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 Processor -- start 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 -- start start 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2019dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.899+0000 7f9f274f1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f2010e090 con 0x7f9f2007dec0 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f2528d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2019dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f2528d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2019dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33132/0 (socket says 172.21.15.87:33132) 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f2528d700 1 -- 172.21.15.87:0/2670882621 learned_addr learned my addr 172.21.15.87:0/2670882621 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f2528d700 1 -- 172.21.15.87:0/2670882621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f2019e1c0 con 0x7f9f2007dec0 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f2528d700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2019dc80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f1c003b20 con 0x7f9f2007dec0 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9f1c003cc0 con 0x7f9f2007dec0 2024-03-21T08:47:43.955 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f2019e450 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9f1c024910 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.900+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f201a1140 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.901+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f9f1c022020 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.901+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f200637c0 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.901+0000 7f9f167fc700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f9f0c0391d0 0x7f9f0c03b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.901+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f9f1c011e00 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.902+0000 7f9f24a8c700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f9f0c0391d0 0x7f9f0c03b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.902+0000 7f9f24a8c700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f9f0c0391d0 0x7f9f0c03b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f10006fd0 tx=0x7f9f10006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:38.906+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9f1c01ad80 con 0x7f9f2007dec0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:39.106+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi087", "addr": "172.21.15.87", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f201043a0 con 0x7f9f0c0391d0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.392+0000 7f9f167fc700 1 -- 172.21.15.87:0/2670882621 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f9f201043a0 con 0x7f9f0c0391d0 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f9f0c0391d0 msgr2=0x7f9f0c03b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:43.956 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f9f0c0391d0 0x7f9f0c03b690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f9f10006fd0 tx=0x7f9f10006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 msgr2=0x7f9f2019dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2019dc80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c000c00 tx=0x7f9f1c011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 shutdown_connections 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f9f0c0391d0 0x7f9f0c03b690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 --2- 172.21.15.87:0/2670882621 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9f2007dec0 0x7f9f2019dc80 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 >> 172.21.15.87:0/2670882621 conn(0x7f9f201030c0 msgr2=0x7f9f20103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 shutdown_connections 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:43.395+0000 7f9f274f1700 1 -- 172.21.15.87:0/2670882621 wait complete. 2024-03-21T08:47:43.957 INFO:teuthology.orchestra.run.smithi087.stdout:Deploying unmanaged mon service... 2024-03-21T08:47:44.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:44 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:44.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:44 smithi087 ceph-mon[32385]: Added host smithi087 2024-03-21T08:47:44.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:44 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5cc80700 1 Processor -- start 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5cc80700 1 -- start start 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5cc80700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b581061d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5cc80700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b581067a0 con 0x7f0b58105db0 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b581061d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b581061d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33148/0 (socket says 172.21.15.87:33148) 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.377+0000 7f0b5659c700 1 -- 172.21.15.87:0/1761141543 learned_addr learned my addr 172.21.15.87:0/1761141543 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5659c700 1 -- 172.21.15.87:0/1761141543 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b58108de0 con 0x7f0b58105db0 2024-03-21T08:47:45.195 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5659c700 1 --2- 172.21.15.87:0/1761141543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b581061d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b480098a0 tx=0x7f0b48009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef204f6527fb699d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5559a700 1 -- 172.21.15.87:0/1761141543 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b48004160 con 0x7f0b58105db0 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5559a700 1 -- 172.21.15.87:0/1761141543 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b48004300 con 0x7f0b58105db0 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5559a700 1 -- 172.21.15.87:0/1761141543 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b48011b50 con 0x7f0b58105db0 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5cc80700 1 -- 172.21.15.87:0/1761141543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 msgr2=0x7f0b581061d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.378+0000 7f0b5cc80700 1 --2- 172.21.15.87:0/1761141543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b581061d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b480098a0 tx=0x7f0b48009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 -- 172.21.15.87:0/1761141543 shutdown_connections 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 --2- 172.21.15.87:0/1761141543 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b581061d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 -- 172.21.15.87:0/1761141543 >> 172.21.15.87:0/1761141543 conn(0x7f0b58101740 msgr2=0x7f0b58103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 -- 172.21.15.87:0/1761141543 shutdown_connections 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 -- 172.21.15.87:0/1761141543 wait complete. 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 Processor -- start 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.379+0000 7f0b5cc80700 1 -- start start 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5cc80700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b5819bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:45.196 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5cc80700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5810c010 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b5819bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b5819bb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33152/0 (socket says 172.21.15.87:33152) 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5659c700 1 -- 172.21.15.87:0/3116210208 learned_addr learned my addr 172.21.15.87:0/3116210208 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5659c700 1 -- 172.21.15.87:0/3116210208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5819c040 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5659c700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b5819bb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009fb0 tx=0x7f0b48011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b480038b0 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5819c2d0 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b48011e50 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0b48019550 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.380+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5819efc0 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.381+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f0b48019850 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.381+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b580637c0 con 0x7f0b58105db0 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.381+0000 7f0b477fe700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0b40039130 0x7f0b4003b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:45.197 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.381+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0b4804d4e0 con 0x7f0b58105db0 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.381+0000 7f0b55d9b700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0b40039130 0x7f0b4003b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.382+0000 7f0b55d9b700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0b40039130 0x7f0b4003b5f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c006fd0 tx=0x7f0b4c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.386+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b48020240 con 0x7f0b58105db0 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.597+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0b58102a30 con 0x7f0b40039130 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.601+0000 7f0b477fe700 1 -- 172.21.15.87:0/3116210208 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0b58102a30 con 0x7f0b40039130 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.603+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0b40039130 msgr2=0x7f0b4003b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.603+0000 7f0b5cc80700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0b40039130 0x7f0b4003b5f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c006fd0 tx=0x7f0b4c006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.603+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 msgr2=0x7f0b5819bb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.603+0000 7f0b5cc80700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b5819bb00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b48009fb0 tx=0x7f0b48011bf0 comp rx=0 tx=0).stop 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.604+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 shutdown_connections 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.604+0000 7f0b5cc80700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f0b40039130 0x7f0b4003b5f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:45.198 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.604+0000 7f0b5cc80700 1 --2- 172.21.15.87:0/3116210208 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b58105db0 0x7f0b5819bb00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:45.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.604+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 >> 172.21.15.87:0/3116210208 conn(0x7f0b58101740 msgr2=0x7f0b58102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:45.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.604+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 shutdown_connections 2024-03-21T08:47:45.199 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:44.604+0000 7f0b5cc80700 1 -- 172.21.15.87:0/3116210208 wait complete. 2024-03-21T08:47:45.199 INFO:teuthology.orchestra.run.smithi087.stdout:Deploying unmanaged mgr service... 2024-03-21T08:47:45.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:45 smithi087 ceph-mon[32385]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:45.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:45 smithi087 ceph-mon[32385]: Saving service mon spec with placement count:5 2024-03-21T08:47:45.775 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:45 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:46.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:46 smithi087 ceph-mon[32385]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:47:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:46 smithi087 ceph-mon[32385]: Saving service mgr spec with placement count:2 2024-03-21T08:47:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:46 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:47.300 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.626+0000 7f16f8811700 1 Processor -- start 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.626+0000 7f16f8811700 1 -- start start 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f8811700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f41083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f8811700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4108990 con 0x7f16f4107fa0 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f41083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f41083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33168/0 (socket says 172.21.15.87:33168) 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f259c700 1 -- 172.21.15.87:0/1655595257 learned_addr learned my addr 172.21.15.87:0/1655595257 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f259c700 1 -- 172.21.15.87:0/1655595257 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4106bb0 con 0x7f16f4107fa0 2024-03-21T08:47:47.301 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f259c700 1 --2- 172.21.15.87:0/1655595257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f41083c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009e80 tx=0x7f16e400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=456c6346882313b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f159a700 1 -- 172.21.15.87:0/1655595257 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e4017070 con 0x7f16f4107fa0 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f159a700 1 -- 172.21.15.87:0/1655595257 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16e400b8c0 con 0x7f16f4107fa0 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.627+0000 7f16f159a700 1 -- 172.21.15.87:0/1655595257 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e40045c0 con 0x7f16f4107fa0 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 -- 172.21.15.87:0/1655595257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 msgr2=0x7f16f41083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 --2- 172.21.15.87:0/1655595257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f41083c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009e80 tx=0x7f16e400b330 comp rx=0 tx=0).stop 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 -- 172.21.15.87:0/1655595257 shutdown_connections 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 --2- 172.21.15.87:0/1655595257 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f41083c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 -- 172.21.15.87:0/1655595257 >> 172.21.15.87:0/1655595257 conn(0x7f16f4101740 msgr2=0x7f16f4103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 -- 172.21.15.87:0/1655595257 shutdown_connections 2024-03-21T08:47:47.302 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.628+0000 7f16f8811700 1 -- 172.21.15.87:0/1655595257 wait complete. 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f8811700 1 Processor -- start 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f8811700 1 -- start start 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f8811700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f4191b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f8811700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f410af70 con 0x7f16f4107fa0 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f4191b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f4191b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:33182/0 (socket says 172.21.15.87:33182) 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f259c700 1 -- 172.21.15.87:0/73015781 learned_addr learned my addr 172.21.15.87:0/73015781 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f259c700 1 -- 172.21.15.87:0/73015781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f4192060 con 0x7f16f4107fa0 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.629+0000 7f16f259c700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f4191b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009e50 tx=0x7f16e400bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e4020020 con 0x7f16f4107fa0 2024-03-21T08:47:47.303 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f16e4003b50 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f41902a0 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f16e4017040 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f41907e0 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f16e4023020 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.630+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f40637c0 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.631+0000 7f16e37fe700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f16dc039130 0x7f16dc03b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.631+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f16e400f050 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.631+0000 7f16f1d9b700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f16dc039130 0x7f16dc03b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.631+0000 7f16f1d9b700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f16dc039130 0x7f16dc03b5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006fd0 tx=0x7f16e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.636+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f16e402a020 con 0x7f16f4107fa0 2024-03-21T08:47:47.304 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.847+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f16f4195930 con 0x7f16dc039130 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.854+0000 7f16e37fe700 1 -- 172.21.15.87:0/73015781 <== mgr.14120 v2:172.21.15.87:6800/2393104803 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f16f4195930 con 0x7f16dc039130 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f16dc039130 msgr2=0x7f16dc03b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f16dc039130 0x7f16dc03b5f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006fd0 tx=0x7f16e8006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 msgr2=0x7f16f4191b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f4191b20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f16e4009e50 tx=0x7f16e400bd30 comp rx=0 tx=0).stop 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 shutdown_connections 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f16dc039130 0x7f16dc03b5f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 --2- 172.21.15.87:0/73015781 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f16f4107fa0 0x7f16f4191b20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.856+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 >> 172.21.15.87:0/73015781 conn(0x7f16f4101740 msgr2=0x7f16f4102630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.857+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 shutdown_connections 2024-03-21T08:47:47.305 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:45.857+0000 7f16f8811700 1 -- 172.21.15.87:0/73015781 wait complete. 2024-03-21T08:47:48.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:47 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:48.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:47 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:49.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:49 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2022777413' entity='client.admin' 2024-03-21T08:47:49.269 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:49 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1c1db0700 1 Processor -- start 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1c1db0700 1 -- start start 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1c1db0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc10a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1c1db0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc07be10 con 0x7ff1bc1085a0 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc10a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc10a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60898/0 (socket says 172.21.15.87:60898) 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.767+0000 7ff1bb7fe700 1 -- 172.21.15.87:0/4277055434 learned_addr learned my addr 172.21.15.87:0/4277055434 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.768+0000 7ff1bb7fe700 1 -- 172.21.15.87:0/4277055434 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc10aed0 con 0x7ff1bc1085a0 2024-03-21T08:47:49.276 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.768+0000 7ff1bb7fe700 1 --2- 172.21.15.87:0/4277055434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc10a990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00098a0 tx=0x7ff1b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffafa518ca4621a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.768+0000 7ff1ba7fc700 1 -- 172.21.15.87:0/4277055434 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b0004160 con 0x7ff1bc1085a0 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.768+0000 7ff1ba7fc700 1 -- 172.21.15.87:0/4277055434 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1b0004300 con 0x7ff1bc1085a0 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.768+0000 7ff1ba7fc700 1 -- 172.21.15.87:0/4277055434 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b0011b20 con 0x7ff1bc1085a0 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 -- 172.21.15.87:0/4277055434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 msgr2=0x7ff1bc10a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 --2- 172.21.15.87:0/4277055434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc10a990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00098a0 tx=0x7ff1b0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 -- 172.21.15.87:0/4277055434 shutdown_connections 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 --2- 172.21.15.87:0/4277055434 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc10a990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 -- 172.21.15.87:0/4277055434 >> 172.21.15.87:0/4277055434 conn(0x7ff1bc101f10 msgr2=0x7ff1bc104350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 -- 172.21.15.87:0/4277055434 shutdown_connections 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.769+0000 7ff1c1db0700 1 -- 172.21.15.87:0/4277055434 wait complete. 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1c1db0700 1 Processor -- start 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1c1db0700 1 -- start start 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1c1db0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc19b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:49.277 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1c1db0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1bc10be80 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc19b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1bb7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc19b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60906/0 (socket says 172.21.15.87:60906) 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1bb7fe700 1 -- 172.21.15.87:0/2022777413 learned_addr learned my addr 172.21.15.87:0/2022777413 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.770+0000 7ff1bb7fe700 1 -- 172.21.15.87:0/2022777413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1bc19bf20 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.771+0000 7ff1bb7fe700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc19b9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00116a0 tx=0x7ff1b0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.771+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b0003a40 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.771+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1bc19c1b0 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.771+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff1b0003be0 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.771+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff1b001a750 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.771+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1bc19c610 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.772+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7ff1b001aa70 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.772+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1bc069250 con 0x7ff1bc1085a0 2024-03-21T08:47:49.278 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.772+0000 7ff1b8ff9700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff1a4039130 0x7ff1a403b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.772+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff1b004d870 con 0x7ff1bc1085a0 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.772+0000 7ff1baffd700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff1a4039130 0x7ff1a403b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.772+0000 7ff1baffd700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff1a4039130 0x7ff1a403b5f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac006fd0 tx=0x7ff1ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:47.777+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff1b0011e20 con 0x7ff1bc1085a0 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.016+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7ff1bc19ef60 con 0x7ff1bc1085a0 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.021+0000 7ff1b8ff9700 1 -- 172.21.15.87:0/2022777413 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7ff1b000b9e0 con 0x7ff1bc1085a0 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff1a4039130 msgr2=0x7ff1a403b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff1a4039130 0x7ff1a403b5f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff1ac006fd0 tx=0x7ff1ac006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 msgr2=0x7ff1bc19b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc19b9e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff1b00116a0 tx=0x7ff1b0011ba0 comp rx=0 tx=0).stop 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 shutdown_connections 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff1a4039130 0x7ff1a403b5f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 --2- 172.21.15.87:0/2022777413 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff1bc1085a0 0x7ff1bc19b9e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:49.279 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.023+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 >> 172.21.15.87:0/2022777413 conn(0x7ff1bc101f10 msgr2=0x7ff1bc102a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:49.280 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.024+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 shutdown_connections 2024-03-21T08:47:49.280 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:48.024+0000 7ff1c1db0700 1 -- 172.21.15.87:0/2022777413 wait complete. 2024-03-21T08:47:50.508 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a9693700 1 Processor -- start 2024-03-21T08:47:50.508 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a9693700 1 -- start start 2024-03-21T08:47:50.508 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a9693700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4106190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:50.508 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a9693700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a4106760 con 0x7ff7a4105d70 2024-03-21T08:47:50.508 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4106190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4106190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60908/0 (socket says 172.21.15.87:60908) 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a2ffd700 1 -- 172.21.15.87:0/3168518702 learned_addr learned my addr 172.21.15.87:0/3168518702 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.698+0000 7ff7a2ffd700 1 -- 172.21.15.87:0/3168518702 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a41068e0 con 0x7ff7a4105d70 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a2ffd700 1 --2- 172.21.15.87:0/3168518702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4106190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff7940098a0 tx=0x7ff794009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179f44e8d44d803c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a1ffb700 1 -- 172.21.15.87:0/3168518702 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff794004160 con 0x7ff7a4105d70 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a1ffb700 1 -- 172.21.15.87:0/3168518702 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff794004300 con 0x7ff7a4105d70 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a1ffb700 1 -- 172.21.15.87:0/3168518702 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff794011b20 con 0x7ff7a4105d70 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a9693700 1 -- 172.21.15.87:0/3168518702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 msgr2=0x7ff7a4106190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:50.509 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a9693700 1 --2- 172.21.15.87:0/3168518702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4106190 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7ff7940098a0 tx=0x7ff794009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a9693700 1 -- 172.21.15.87:0/3168518702 shutdown_connections 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a9693700 1 --2- 172.21.15.87:0/3168518702 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4106190 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.699+0000 7ff7a9693700 1 -- 172.21.15.87:0/3168518702 >> 172.21.15.87:0/3168518702 conn(0x7ff7a41016d0 msgr2=0x7ff7a4103af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.700+0000 7ff7a9693700 1 -- 172.21.15.87:0/3168518702 shutdown_connections 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.700+0000 7ff7a9693700 1 -- 172.21.15.87:0/3168518702 wait complete. 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.700+0000 7ff7a9693700 1 Processor -- start 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.700+0000 7ff7a9693700 1 -- start start 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.700+0000 7ff7a9693700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4108620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.700+0000 7ff7a9693700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a410c020 con 0x7ff7a4105d70 2024-03-21T08:47:50.510 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4108620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4108620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60912/0 (socket says 172.21.15.87:60912) 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a2ffd700 1 -- 172.21.15.87:0/1047537765 learned_addr learned my addr 172.21.15.87:0/1047537765 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a2ffd700 1 -- 172.21.15.87:0/1047537765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4108b60 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a2ffd700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4108620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7940117e0 tx=0x7ff794011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff794003bf0 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff79401a910 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7a4106da0 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff7940249a0 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.701+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7a4107260 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.702+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7ff794022020 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.702+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7a40637c0 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.702+0000 7ff78bfff700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff78c039130 0x7ff78c03b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.702+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff79404d460 con 0x7ff7a4105d70 2024-03-21T08:47:50.511 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.702+0000 7ff7a27fc700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff78c039130 0x7ff78c03b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.703+0000 7ff7a27fc700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff78c039130 0x7ff78c03b5f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff798006fd0 tx=0x7ff798006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.709+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff794011e20 con 0x7ff7a4105d70 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.914+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7ff7a41075a0 con 0x7ff7a4105d70 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.916+0000 7ff78bfff700 1 -- 172.21.15.87:0/1047537765 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7ff7a41075a0 con 0x7ff7a4105d70 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff78c039130 msgr2=0x7ff78c03b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff78c039130 0x7ff78c03b5f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff798006fd0 tx=0x7ff798006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 msgr2=0x7ff7a4108620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4108620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff7940117e0 tx=0x7ff794011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 shutdown_connections 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ff78c039130 0x7ff78c03b5f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 --2- 172.21.15.87:0/1047537765 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7a4105d70 0x7ff7a4108620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.918+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 >> 172.21.15.87:0/1047537765 conn(0x7ff7a41016d0 msgr2=0x7ff7a4102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.919+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 shutdown_connections 2024-03-21T08:47:50.512 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:49.919+0000 7ff7a9693700 1 -- 172.21.15.87:0/1047537765 wait complete. 2024-03-21T08:47:50.513 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling the dashboard module... 2024-03-21T08:47:51.063 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:50 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1047537765' entity='client.admin' 2024-03-21T08:47:51.063 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:50 smithi087 ceph-mon[32385]: from='mgr.14120 172.21.15.87:0/292810794' entity='mgr.x' 2024-03-21T08:47:52.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:51 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2975229001' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-21T08:47:52.159 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: ignoring --setuser ceph since I am not root 2024-03-21T08:47:52.159 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: ignoring --setgroup ceph since I am not root 2024-03-21T08:47:52.586 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:52 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:52.491+0000 7fbc051a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:47:52.586 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:52 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:52.586+0000 7fbc051a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.925+0000 7f02d7048700 1 Processor -- start 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.925+0000 7f02d7048700 1 -- start start 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d7048700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d0106780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d7048700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d007be50 con 0x7f02d0106360 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d4de4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d0106780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d4de4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d0106780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60928/0 (socket says 172.21.15.87:60928) 2024-03-21T08:47:52.613 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d4de4700 1 -- 172.21.15.87:0/2163138 learned_addr learned my addr 172.21.15.87:0/2163138 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d4de4700 1 -- 172.21.15.87:0/2163138 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d0106cc0 con 0x7f02d0106360 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.926+0000 7f02d4de4700 1 --2- 172.21.15.87:0/2163138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d0106780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02c40098a0 tx=0x7f02c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60ff09188b61d652 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.927+0000 7f02cf7fe700 1 -- 172.21.15.87:0/2163138 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4004160 con 0x7f02d0106360 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.927+0000 7f02cf7fe700 1 -- 172.21.15.87:0/2163138 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c4004300 con 0x7f02d0106360 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.927+0000 7f02cf7fe700 1 -- 172.21.15.87:0/2163138 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4011b20 con 0x7f02d0106360 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.927+0000 7f02d7048700 1 -- 172.21.15.87:0/2163138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 msgr2=0x7f02d0106780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 --2- 172.21.15.87:0/2163138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d0106780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f02c40098a0 tx=0x7f02c4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 -- 172.21.15.87:0/2163138 shutdown_connections 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 --2- 172.21.15.87:0/2163138 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d0106780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 -- 172.21.15.87:0/2163138 >> 172.21.15.87:0/2163138 conn(0x7f02d0101f40 msgr2=0x7f02d0104360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 -- 172.21.15.87:0/2163138 shutdown_connections 2024-03-21T08:47:52.614 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 -- 172.21.15.87:0/2163138 wait complete. 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 Processor -- start 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 -- start start 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d019dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.928+0000 7f02d7048700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02d0107d90 con 0x7f02d0106360 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02d4de4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d019dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02d4de4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d019dbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60930/0 (socket says 172.21.15.87:60930) 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02d4de4700 1 -- 172.21.15.87:0/2975229001 learned_addr learned my addr 172.21.15.87:0/2975229001 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02d4de4700 1 -- 172.21.15.87:0/2975229001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d019e0f0 con 0x7f02d0106360 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02d4de4700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d019dbb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02c40117e0 tx=0x7f02c4011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4003bf0 con 0x7f02d0106360 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02d019e380 con 0x7f02d0106360 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f02c401a910 con 0x7f02d0106360 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.929+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02c4024970 con 0x7f02d0106360 2024-03-21T08:47:52.615 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.930+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02d019e7e0 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.931+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02d00637c0 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.931+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 8) v1 ==== 46294+0+0 (secure 0 0 0) 0x7f02c4022020 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.931+0000 7f02cdffb700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f02c0039180 0x7f02c003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.931+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f02c404d490 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.931+0000 7f02cffff700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f02c0039180 0x7f02c003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.932+0000 7f02cffff700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f02c0039180 0x7f02c003b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02bc006fd0 tx=0x7f02bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:50.940+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f02c4011e20 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.214+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f02d002de70 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.920+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f02d002de70 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.921+0000 7f02cdffb700 1 -- 172.21.15.87:0/2975229001 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f02c402d910 con 0x7f02d0106360 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f02c0039180 msgr2=0x7f02c003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f02c0039180 0x7f02c003b640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f02bc006fd0 tx=0x7f02bc006e40 comp rx=0 tx=0).stop 2024-03-21T08:47:52.616 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 msgr2=0x7f02d019dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d019dbb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f02c40117e0 tx=0x7f02c4011c10 comp rx=0 tx=0).stop 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 shutdown_connections 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f02c0039180 0x7f02c003b640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 --2- 172.21.15.87:0/2975229001 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f02d0106360 0x7f02d019dbb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 >> 172.21.15.87:0/2975229001 conn(0x7f02d0101f40 msgr2=0x7f02d0194820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 shutdown_connections 2024-03-21T08:47:52.617 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:51.923+0000 7f02d7048700 1 -- 172.21.15.87:0/2975229001 wait complete. 2024-03-21T08:47:53.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:52 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2975229001' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-21T08:47:53.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:52 smithi087 ceph-mon[32385]: mgrmap e9: x(active, since 20s) 2024-03-21T08:47:53.603 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:53.267+0000 7fbc051a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:47:53.603 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:53.361+0000 7fbc051a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:47:53.603 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:53.509+0000 7fbc051a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:47:53.603 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:53.605+0000 7fbc051a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:47:53.908 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:53.688+0000 7fbc051a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:47:53.929 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.069+0000 7f536eaf0700 1 Processor -- start 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.069+0000 7f536eaf0700 1 -- start start 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536eaf0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536807c320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:53.930 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536eaf0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536807c8f0 con 0x7f536807dec0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536c88c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536807c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536c88c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536807c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60948/0 (socket says 172.21.15.87:60948) 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536c88c700 1 -- 172.21.15.87:0/4054361379 learned_addr learned my addr 172.21.15.87:0/4054361379 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536c88c700 1 -- 172.21.15.87:0/4054361379 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536807ca70 con 0x7f536807dec0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f536c88c700 1 --2- 172.21.15.87:0/4054361379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536807c320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098a0 tx=0x7f535c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fde8175fa910b855 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f53677fe700 1 -- 172.21.15.87:0/4054361379 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c004160 con 0x7f536807dec0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f53677fe700 1 -- 172.21.15.87:0/4054361379 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f535c004300 con 0x7f536807dec0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.070+0000 7f53677fe700 1 -- 172.21.15.87:0/4054361379 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c011b20 con 0x7f536807dec0 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 -- 172.21.15.87:0/4054361379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 msgr2=0x7f536807c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 --2- 172.21.15.87:0/4054361379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536807c320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f535c0098a0 tx=0x7f535c009bb0 comp rx=0 tx=0).stop 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 -- 172.21.15.87:0/4054361379 shutdown_connections 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 --2- 172.21.15.87:0/4054361379 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536807c320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:53.931 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 -- 172.21.15.87:0/4054361379 >> 172.21.15.87:0/4054361379 conn(0x7f5368103050 msgr2=0x7f5368105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 -- 172.21.15.87:0/4054361379 shutdown_connections 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.071+0000 7f536eaf0700 1 -- 172.21.15.87:0/4054361379 wait complete. 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536eaf0700 1 Processor -- start 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536eaf0700 1 -- start start 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536eaf0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536819dc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536eaf0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f536810e0f0 con 0x7f536807dec0 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536c88c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536819dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536c88c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536819dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60954/0 (socket says 172.21.15.87:60954) 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536c88c700 1 -- 172.21.15.87:0/1795612011 learned_addr learned my addr 172.21.15.87:0/1795612011 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536c88c700 1 -- 172.21.15.87:0/1795612011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f536819e1a0 con 0x7f536807dec0 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f536c88c700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536819dc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f535c0116a0 tx=0x7f535c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.072+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c003a40 con 0x7f536807dec0 2024-03-21T08:47:53.932 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.073+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f535c003be0 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.073+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f536819e430 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.073+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f535c01a720 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.073+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f536819e890 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.073+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f535c01aa20 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.074+0000 7f5365ffb700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 0x7f535803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.074+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f535c04d840 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.074+0000 7f5367fff700 1 -- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 msgr2=0x7f535803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.074+0000 7f5367fff700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 0x7f535803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.074+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5368069250 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.079+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f535c011e20 con 0x7f536807dec0 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.275+0000 7f5367fff700 1 -- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 msgr2=0x7f535803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:47:53.933 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.275+0000 7f5367fff700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 0x7f535803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.358+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f53681a1220 con 0x7f536807dec0 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.359+0000 7f5365ffb700 1 -- 172.21.15.87:0/1795612011 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f535c00ba00 con 0x7f536807dec0 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 msgr2=0x7f535803b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 0x7f535803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 msgr2=0x7f536819dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536819dc60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f535c0116a0 tx=0x7f535c011ba0 comp rx=0 tx=0).stop 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 shutdown_connections 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7f5358039180 0x7f535803b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 --2- 172.21.15.87:0/1795612011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f536807dec0 0x7f536819dc60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 >> 172.21.15.87:0/1795612011 conn(0x7f5368103050 msgr2=0x7f5368103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:47:53.934 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 shutdown_connections 2024-03-21T08:47:53.935 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:53.361+0000 7f536eaf0700 1 -- 172.21.15.87:0/1795612011 wait complete. 2024-03-21T08:47:53.935 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for the mgr to restart... 2024-03-21T08:47:53.935 INFO:teuthology.orchestra.run.smithi087.stdout:Waiting for mgr epoch 9... 2024-03-21T08:47:54.170 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1795612011' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-21T08:47:54.170 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:53.954+0000 7fbc051a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:47:54.171 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:54.067+0000 7fbc051a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:47:54.171 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:54.152+0000 7fbc051a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:47:54.427 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:54.250+0000 7fbc051a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:47:54.908 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:54.428+0000 7fbc051a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:47:55.658 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:55.375+0000 7fbc051a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:47:55.659 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:55.504+0000 7fbc051a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:47:56.115 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:55.832+0000 7fbc051a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:47:56.115 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:55.921+0000 7fbc051a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:47:56.116 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:56.117+0000 7fbc051a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:47:56.408 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:56.300+0000 7fbc051a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:47:56.840 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:56.586+0000 7fbc051a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:47:56.840 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:56.841+0000 7fbc051a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:47:57.158 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:57.076+0000 7fbc051a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:47:58.158 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:57.876+0000 7fbc051a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:47:58.755 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:58.439+0000 7fbc051a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:47:58.755 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:58.537+0000 7fbc051a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:47:58.755 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:47:58.756+0000 7fbc051a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:47:59.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: Active manager daemon x restarted 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: Activating manager daemon x 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: osdmap e3: 0 total, 0 up, 0 in 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: mgrmap e10: x(active, starting, since 0.0034661s) 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: Manager daemon x is now available 2024-03-21T08:47:59.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:00.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-03-21T08:48:00.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-03-21T08:48:00.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:47:59 smithi087 ceph-mon[32385]: mgrmap e11: x(active, since 1.00577s) 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout { 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout } 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.386+0000 7ffb54c6b700 1 Processor -- start 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb54c6b700 1 -- start start 2024-03-21T08:48:00.708 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb54c6b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb50107eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb54c6b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb50108480 con 0x7ffb50107a90 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb4e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb50107eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb4e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb50107eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60962/0 (socket says 172.21.15.87:60962) 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb4e59c700 1 -- 172.21.15.87:0/3349344210 learned_addr learned my addr 172.21.15.87:0/3349344210 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.387+0000 7ffb4e59c700 1 -- 172.21.15.87:0/3349344210 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50108c90 con 0x7ffb50107a90 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb4e59c700 1 --2- 172.21.15.87:0/3349344210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb50107eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb440098a0 tx=0x7ffb44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f3161e2495d949f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb4d59a700 1 -- 172.21.15.87:0/3349344210 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb44004160 con 0x7ffb50107a90 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb4d59a700 1 -- 172.21.15.87:0/3349344210 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb44004300 con 0x7ffb50107a90 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb4d59a700 1 -- 172.21.15.87:0/3349344210 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb44011b20 con 0x7ffb50107a90 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3349344210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 msgr2=0x7ffb50107eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3349344210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb50107eb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffb440098a0 tx=0x7ffb44009bb0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.709 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3349344210 shutdown_connections 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.388+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3349344210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb50107eb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.389+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3349344210 >> 172.21.15.87:0/3349344210 conn(0x7ffb50103050 msgr2=0x7ffb50105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.389+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3349344210 shutdown_connections 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.389+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3349344210 wait complete. 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.389+0000 7ffb54c6b700 1 Processor -- start 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.389+0000 7ffb54c6b700 1 -- start start 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb54c6b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb5019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb54c6b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5010c040 con 0x7ffb50107a90 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb4e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb5019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb4e59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb5019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:60968/0 (socket says 172.21.15.87:60968) 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb4e59c700 1 -- 172.21.15.87:0/3494944787 learned_addr learned my addr 172.21.15.87:0/3494944787 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb4e59c700 1 -- 172.21.15.87:0/3494944787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5019bf80 con 0x7ffb50107a90 2024-03-21T08:48:00.710 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb4e59c700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb5019ba40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb440116a0 tx=0x7ffb44011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb44003a40 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffb44003be0 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5019c210 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffb4401a720 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.390+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5019c670 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.391+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 9) v1 ==== 46307+0+0 (secure 0 0 0) 0x7ffb4401aa20 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.391+0000 7ffb3f7fe700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.391+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 --> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb3803bd50 con 0x7ffb38039180 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.391+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffb4404d970 con 0x7ffb50107a90 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.391+0000 7ffb4dd9b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 msgr2=0x7ffb3803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.391+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.592+0000 7ffb4dd9b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 msgr2=0x7ffb3803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.592+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-21T08:48:00.711 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.993+0000 7ffb4dd9b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 msgr2=0x7ffb3803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:54.993+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:55.794+0000 7ffb4dd9b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 msgr2=0x7ffb3803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:55.794+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:57.395+0000 7ffb4dd9b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 msgr2=0x7ffb3803b640 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.87:6800/2393104803 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:57.395+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:58.772+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mgrmap(e 10) v1 ==== 46074+0+0 (secure 0 0 0) 0x7ffb44024430 con 0x7ffb50107a90 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:58.772+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 msgr2=0x7ffb3803b640 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:58.772+0000 7ffb3f7fe700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.773+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 11) v1 ==== 46201+0+0 (secure 0 0 0) 0x7ffb4404f6b0 con 0x7ffb50107a90 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.774+0000 7ffb3f7fe700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffb3803cd20 0x7ffb3803f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.774+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ffb44016e70 con 0x7ffb3803cd20 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.774+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffb3803cd20 0x7ffb3803f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.774+0000 7ffb4dd9b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffb3803cd20 0x7ffb3803f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb40003a10 tx=0x7ffb400092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:00.712 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.775+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7ffb44016e70 con 0x7ffb3803cd20 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.781+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ffb50069250 con 0x7ffb3803cd20 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb3f7fe700 1 -- 172.21.15.87:0/3494944787 <== mgr.14152 v2:172.21.15.87:6800/1005139006 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7ffb50069250 con 0x7ffb3803cd20 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffb3803cd20 msgr2=0x7ffb3803f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffb3803cd20 0x7ffb3803f110 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ffb40003a10 tx=0x7ffb400092b0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 msgr2=0x7ffb5019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb5019ba40 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ffb440116a0 tx=0x7ffb44011ba0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 shutdown_connections 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffb3803cd20 0x7ffb3803f110 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb50107a90 0x7ffb5019ba40 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 --2- 172.21.15.87:0/3494944787 >> [v2:172.21.15.87:6800/2393104803,v1:172.21.15.87:6801/2393104803] conn(0x7ffb38039180 0x7ffb3803b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 >> 172.21.15.87:0/3494944787 conn(0x7ffb50103050 msgr2=0x7ffb50103b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 shutdown_connections 2024-03-21T08:48:00.713 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:47:59.782+0000 7ffb54c6b700 1 -- 172.21.15.87:0/3494944787 wait complete. 2024-03-21T08:48:00.714 INFO:teuthology.orchestra.run.smithi087.stdout:mgr epoch 9 is available 2024-03-21T08:48:00.714 INFO:teuthology.orchestra.run.smithi087.stdout:Generating a dashboard self-signed certificate... 2024-03-21T08:48:01.034 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:00 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:59] ENGINE Bus STARTING 2024-03-21T08:48:01.035 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:00 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:59] ENGINE Serving on http://172.21.15.87:8765 2024-03-21T08:48:01.035 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:00 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:59] ENGINE Serving on https://172.21.15.87:7150 2024-03-21T08:48:01.035 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:00 smithi087 ceph-mon[32385]: [21/Mar/2024:08:47:59] ENGINE Bus STARTED 2024-03-21T08:48:01.035 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:00 smithi087 ceph-mon[32385]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-21T08:48:01.035 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:00 smithi087 ceph-mon[32385]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-21T08:48:02.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:01 smithi087 ceph-mon[32385]: mgrmap e12: x(active, since 2s) 2024-03-21T08:48:02.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:02.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:02.080 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:02.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-21T08:48:02.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b6beff700 1 Processor -- start 2024-03-21T08:48:02.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b6beff700 1 -- start start 2024-03-21T08:48:02.088 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b6beff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6407c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b6beff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6407c840 con 0x7f7b6407de50 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b69c9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6407c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b69c9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6407c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49268/0 (socket says 172.21.15.87:49268) 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.136+0000 7f7b69c9b700 1 -- 172.21.15.87:0/227735485 learned_addr learned my addr 172.21.15.87:0/227735485 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b69c9b700 1 -- 172.21.15.87:0/227735485 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6407c9c0 con 0x7f7b6407de50 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b69c9b700 1 --2- 172.21.15.87:0/227735485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6407c300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b600098a0 tx=0x7f7b60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7511ec29830940d3 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b68c99700 1 -- 172.21.15.87:0/227735485 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b60004160 con 0x7f7b6407de50 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b68c99700 1 -- 172.21.15.87:0/227735485 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b60004300 con 0x7f7b6407de50 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b68c99700 1 -- 172.21.15.87:0/227735485 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b60011b20 con 0x7f7b6407de50 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b6beff700 1 -- 172.21.15.87:0/227735485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 msgr2=0x7f7b6407c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:02.089 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b6beff700 1 --2- 172.21.15.87:0/227735485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6407c300 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f7b600098a0 tx=0x7f7b60009bb0 comp rx=0 tx=0).stop 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b6beff700 1 -- 172.21.15.87:0/227735485 shutdown_connections 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b6beff700 1 --2- 172.21.15.87:0/227735485 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6407c300 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.137+0000 7f7b6beff700 1 -- 172.21.15.87:0/227735485 >> 172.21.15.87:0/227735485 conn(0x7f7b64103130 msgr2=0x7f7b64105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b6beff700 1 -- 172.21.15.87:0/227735485 shutdown_connections 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b6beff700 1 -- 172.21.15.87:0/227735485 wait complete. 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b6beff700 1 Processor -- start 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b6beff700 1 -- start start 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b6beff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6419dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b6beff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6410e110 con 0x7f7b6407de50 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b69c9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6419dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:02.090 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b69c9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6419dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49270/0 (socket says 172.21.15.87:49270) 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.138+0000 7f7b69c9b700 1 -- 172.21.15.87:0/1494214773 learned_addr learned my addr 172.21.15.87:0/1494214773 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b69c9b700 1 -- 172.21.15.87:0/1494214773 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6419e210 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b69c9b700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6419dcd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b600116a0 tx=0x7f7b60011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b60003ac0 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7b60003c60 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6419e4a0 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7b6001a7d0 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.139+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6419e900 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.140+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f7b6001aad0 con 0x7f7b6407de50 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.140+0000 7f7b5affd700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7b50039180 0x7f7b5003b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:02.091 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.140+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7b6004d880 con 0x7f7b6407de50 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.140+0000 7f7b6949a700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7b50039180 0x7f7b5003b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.140+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b64069250 con 0x7f7b6407de50 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.140+0000 7f7b6949a700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7b50039180 0x7f7b5003b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006fd0 tx=0x7f7b54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.145+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7b60011e20 con 0x7f7b6407de50 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.356+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b641042f0 con 0x7f7b50039180 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.459+0000 7f7b5affd700 1 -- 172.21.15.87:0/1494214773 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f7b641042f0 con 0x7f7b50039180 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7b50039180 msgr2=0x7f7b5003b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7b50039180 0x7f7b5003b640 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7b54006fd0 tx=0x7f7b54006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 msgr2=0x7f7b6419dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:02.092 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6419dcd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7b600116a0 tx=0x7f7b60011c10 comp rx=0 tx=0).stop 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 shutdown_connections 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7b50039180 0x7f7b5003b640 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 --2- 172.21.15.87:0/1494214773 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7b6407de50 0x7f7b6419dcd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 >> 172.21.15.87:0/1494214773 conn(0x7f7b64103130 msgr2=0x7f7b64103be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 shutdown_connections 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:01.461+0000 7f7b6beff700 1 -- 172.21.15.87:0/1494214773 wait complete. 2024-03-21T08:48:02.093 INFO:teuthology.orchestra.run.smithi087.stdout:Creating initial admin user... 2024-03-21T08:48:03.088 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:02 smithi087 ceph-mon[32385]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:48:03.089 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$bml30D/A9zt4ctqsqFXm8eym23xDrvyCAbuP9iBI.Oc4ZHAkwMTHC", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1711010883, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d8e9c700 1 Processor -- start 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d8e9c700 1 -- start start 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d8e9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d41067b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d8e9c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d407be50 con 0x7f15d4106390 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d41067b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d41067b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49282/0 (socket says 172.21.15.87:49282) 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.566+0000 7f15d259c700 1 -- 172.21.15.87:0/829272464 learned_addr learned my addr 172.21.15.87:0/829272464 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.567+0000 7f15d259c700 1 -- 172.21.15.87:0/829272464 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d4106cf0 con 0x7f15d4106390 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.567+0000 7f15d259c700 1 --2- 172.21.15.87:0/829272464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d41067b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15c40098a0 tx=0x7f15c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e61391910529050d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.567+0000 7f15d159a700 1 -- 172.21.15.87:0/829272464 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c4004160 con 0x7f15d4106390 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.567+0000 7f15d159a700 1 -- 172.21.15.87:0/829272464 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c4004300 con 0x7f15d4106390 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.567+0000 7f15d159a700 1 -- 172.21.15.87:0/829272464 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c4011b20 con 0x7f15d4106390 2024-03-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 -- 172.21.15.87:0/829272464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 msgr2=0x7f15d41067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 --2- 172.21.15.87:0/829272464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d41067b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f15c40098a0 tx=0x7f15c4009bb0 comp rx=0 tx=0).stop 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 -- 172.21.15.87:0/829272464 shutdown_connections 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 --2- 172.21.15.87:0/829272464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d41067b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 -- 172.21.15.87:0/829272464 >> 172.21.15.87:0/829272464 conn(0x7f15d4101f50 msgr2=0x7f15d4104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 -- 172.21.15.87:0/829272464 shutdown_connections 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 -- 172.21.15.87:0/829272464 wait complete. 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 Processor -- start 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.568+0000 7f15d8e9c700 1 -- start start 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d8e9c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d419dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d8e9c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d4107dc0 con 0x7f15d4106390 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d419dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d259c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d419dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49284/0 (socket says 172.21.15.87:49284) 2024-03-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d259c700 1 -- 172.21.15.87:0/1797397541 learned_addr learned my addr 172.21.15.87:0/1797397541 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d259c700 1 -- 172.21.15.87:0/1797397541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d419e180 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d259c700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d419dc40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15c4011b60 tx=0x7f15c4011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c4003b60 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f15c401a910 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d419e410 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.569+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f15c4024930 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.570+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d419e870 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.571+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f15c4022020 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.571+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d4069250 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.571+0000 7f15c37fe700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f15bc039230 0x7f15bc03b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.571+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f15c404d640 con 0x7f15d4106390 2024-03-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.571+0000 7f15d1d9b700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f15bc039230 0x7f15bc03b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.571+0000 7f15d1d9b700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f15bc039230 0x7f15bc03b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15c8006fd0 tx=0x7f15c8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.576+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f15c400bc70 con 0x7f15d4106390 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:02.799+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f15d4197760 con 0x7f15bc039230 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.025+0000 7f15c37fe700 1 -- 172.21.15.87:0/1797397541 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f15d4197760 con 0x7f15bc039230 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f15bc039230 msgr2=0x7f15bc03b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f15bc039230 0x7f15bc03b6f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f15c8006fd0 tx=0x7f15c8006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 msgr2=0x7f15d419dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d419dc40 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f15c4011b60 tx=0x7f15c4011c40 comp rx=0 tx=0).stop 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 shutdown_connections 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f15bc039230 0x7f15bc03b6f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 --2- 172.21.15.87:0/1797397541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f15d4106390 0x7f15d419dc40 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 >> 172.21.15.87:0/1797397541 conn(0x7f15d4101f50 msgr2=0x7f15d41948a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 shutdown_connections 2024-03-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:03.027+0000 7f15d8e9c700 1 -- 172.21.15.87:0/1797397541 wait complete. 2024-03-21T08:48:03.650 INFO:teuthology.orchestra.run.smithi087.stdout:Fetching dashboard port number... 2024-03-21T08:48:03.892 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:03 smithi087 ceph-mon[32385]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:48:03.892 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:05.034 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stdout 8443 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f102e861700 1 Processor -- start 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f102e861700 1 -- start start 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f102e861700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f1028107e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f102e861700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028108460 con 0x7f1028107a70 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f1027fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f1028107e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f1027fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f1028107e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49294/0 (socket says 172.21.15.87:49294) 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f1027fff700 1 -- 172.21.15.87:0/225733025 learned_addr learned my addr 172.21.15.87:0/225733025 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.110+0000 7f1027fff700 1 -- 172.21.15.87:0/225733025 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1028108c70 con 0x7f1028107a70 2024-03-21T08:48:05.035 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f1027fff700 1 --2- 172.21.15.87:0/225733025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f1028107e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10180098a0 tx=0x7f1018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1816793c79184e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f1026ffd700 1 -- 172.21.15.87:0/225733025 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1018004160 con 0x7f1028107a70 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f1026ffd700 1 -- 172.21.15.87:0/225733025 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1018004300 con 0x7f1028107a70 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f1026ffd700 1 -- 172.21.15.87:0/225733025 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1018011b20 con 0x7f1028107a70 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 -- 172.21.15.87:0/225733025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 msgr2=0x7f1028107e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 --2- 172.21.15.87:0/225733025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f1028107e90 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f10180098a0 tx=0x7f1018009bb0 comp rx=0 tx=0).stop 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 -- 172.21.15.87:0/225733025 shutdown_connections 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 --2- 172.21.15.87:0/225733025 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f1028107e90 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:05.036 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 -- 172.21.15.87:0/225733025 >> 172.21.15.87:0/225733025 conn(0x7f1028103050 msgr2=0x7f1028105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 -- 172.21.15.87:0/225733025 shutdown_connections 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.111+0000 7f102e861700 1 -- 172.21.15.87:0/225733025 wait complete. 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f102e861700 1 Processor -- start 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f102e861700 1 -- start start 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f102e861700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f102819b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f102e861700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f102810c020 con 0x7f1028107a70 2024-03-21T08:48:05.037 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f1027fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f102819b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f1027fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f102819b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49298/0 (socket says 172.21.15.87:49298) 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f1027fff700 1 -- 172.21.15.87:0/2710801471 learned_addr learned my addr 172.21.15.87:0/2710801471 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.112+0000 7f1027fff700 1 -- 172.21.15.87:0/2710801471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f102819bf20 con 0x7f1028107a70 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f1027fff700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f102819b9e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10180117e0 tx=0x7f1018011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1018003bf0 con 0x7f1028107a70 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f101801a910 con 0x7f1028107a70 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f102819c1b0 con 0x7f1028107a70 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1018024970 con 0x7f1028107a70 2024-03-21T08:48:05.038 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f102819c610 con 0x7f1028107a70 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.113+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 12) v1 ==== 46307+0+0 (secure 0 0 0) 0x7f1018022020 con 0x7f1028107a70 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.114+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10280637c0 con 0x7f1028107a70 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.114+0000 7f10257fa700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1010039230 0x7f101003b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.114+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f101804d7b0 con 0x7f1028107a70 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.114+0000 7f10277fe700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1010039230 0x7f101003b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.114+0000 7f10277fe700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1010039230 0x7f101003b6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f101c006fd0 tx=0x7f101c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.119+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f101800bd80 con 0x7f1028107a70 2024-03-21T08:48:05.039 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.319+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f102819efe0 con 0x7f1028107a70 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.319+0000 7f10257fa700 1 -- 172.21.15.87:0/2710801471 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f101800b5a0 con 0x7f1028107a70 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.321+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1010039230 msgr2=0x7f101003b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.321+0000 7f102e861700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1010039230 0x7f101003b6f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f101c006fd0 tx=0x7f101c006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.321+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 msgr2=0x7f102819b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.321+0000 7f102e861700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f102819b9e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f10180117e0 tx=0x7f1018011ba0 comp rx=0 tx=0).stop 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.322+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 shutdown_connections 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.322+0000 7f102e861700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1010039230 0x7f101003b6f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:05.040 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.322+0000 7f102e861700 1 --2- 172.21.15.87:0/2710801471 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1028107a70 0x7f102819b9e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:05.041 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.322+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 >> 172.21.15.87:0/2710801471 conn(0x7f1028103050 msgr2=0x7f1028103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:05.041 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.322+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 shutdown_connections 2024-03-21T08:48:05.041 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:04.322+0000 7f102e861700 1 -- 172.21.15.87:0/2710801471 wait complete. 2024-03-21T08:48:05.041 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-03-21T08:48:05.042 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout disabled 2024-03-21T08:48:05.045 INFO:teuthology.orchestra.run.smithi087.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-03-21T08:48:05.046 INFO:teuthology.orchestra.run.smithi087.stdout:systemctl: stdout inactive 2024-03-21T08:48:05.046 INFO:teuthology.orchestra.run.smithi087.stdout:firewalld.service is not enabled 2024-03-21T08:48:05.046 INFO:teuthology.orchestra.run.smithi087.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-21T08:48:05.047 INFO:teuthology.orchestra.run.smithi087.stdout:Ceph Dashboard is now available at: 2024-03-21T08:48:05.047 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:05.047 INFO:teuthology.orchestra.run.smithi087.stdout: URL: https://smithi087.front.sepia.ceph.com:8443/ 2024-03-21T08:48:05.048 INFO:teuthology.orchestra.run.smithi087.stdout: User: admin 2024-03-21T08:48:05.048 INFO:teuthology.orchestra.run.smithi087.stdout: Password: 90apkrhsve 2024-03-21T08:48:05.048 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:05.048 INFO:teuthology.orchestra.run.smithi087.stdout:Saving cluster configuration to /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config directory 2024-03-21T08:48:05.049 INFO:teuthology.orchestra.run.smithi087.stdout:Enabling autotune for osd_memory_target 2024-03-21T08:48:05.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:04 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2710801471' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-21T08:48:05.994 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:05 smithi087 ceph-mon[32385]: mgrmap e13: x(active, since 6s) 2024-03-21T08:48:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.471+0000 7fc653bf1700 1 Processor -- start 2024-03-21T08:48:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc653bf1700 1 -- start start 2024-03-21T08:48:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc653bf1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c1083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc653bf1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c108990 con 0x7fc64c107fa0 2024-03-21T08:48:06.406 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc65198d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c1083c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc65198d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c1083c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49314/0 (socket says 172.21.15.87:49314) 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc65198d700 1 -- 172.21.15.87:0/3683558299 learned_addr learned my addr 172.21.15.87:0/3683558299 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.472+0000 7fc65198d700 1 -- 172.21.15.87:0/3683558299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c106bb0 con 0x7fc64c107fa0 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc65198d700 1 --2- 172.21.15.87:0/3683558299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c1083c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6480098a0 tx=0x7fc648009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5697072cd055796d server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc65098b700 1 -- 172.21.15.87:0/3683558299 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc648004160 con 0x7fc64c107fa0 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc65098b700 1 -- 172.21.15.87:0/3683558299 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc648004300 con 0x7fc64c107fa0 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc65098b700 1 -- 172.21.15.87:0/3683558299 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc648011b20 con 0x7fc64c107fa0 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 -- 172.21.15.87:0/3683558299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 msgr2=0x7fc64c1083c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 --2- 172.21.15.87:0/3683558299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c1083c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc6480098a0 tx=0x7fc648009bb0 comp rx=0 tx=0).stop 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 -- 172.21.15.87:0/3683558299 shutdown_connections 2024-03-21T08:48:06.407 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 --2- 172.21.15.87:0/3683558299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c1083c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 -- 172.21.15.87:0/3683558299 >> 172.21.15.87:0/3683558299 conn(0x7fc64c101740 msgr2=0x7fc64c103b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 -- 172.21.15.87:0/3683558299 shutdown_connections 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.473+0000 7fc653bf1700 1 -- 172.21.15.87:0/3683558299 wait complete. 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc653bf1700 1 Processor -- start 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc653bf1700 1 -- start start 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc653bf1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc653bf1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c10af70 con 0x7fc64c107fa0 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc65198d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc65198d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49322/0 (socket says 172.21.15.87:49322) 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc65198d700 1 -- 172.21.15.87:0/157340932 learned_addr learned my addr 172.21.15.87:0/157340932 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.474+0000 7fc65198d700 1 -- 172.21.15.87:0/157340932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c19c070 con 0x7fc64c107fa0 2024-03-21T08:48:06.408 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.475+0000 7fc65198d700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c19bb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6480116a0 tx=0x7fc648011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.475+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc648003a40 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.475+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c19c300 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.475+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc648003be0 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.475+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc64801a720 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.475+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c19c760 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.476+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fc648024430 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.476+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc64c0637c0 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.476+0000 7fc642ffd700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc638039220 0x7fc63803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.476+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc64804d980 con 0x7fc64c107fa0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.476+0000 7fc65118c700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc638039220 0x7fc63803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:06.409 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.476+0000 7fc65118c700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc638039220 0x7fc63803b6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006fd0 tx=0x7fc63c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:06.410 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.481+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc648011e20 con 0x7fc64c107fa0 2024-03-21T08:48:06.410 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.681+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7fc64c062da0 con 0x7fc64c107fa0 2024-03-21T08:48:06.410 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.682+0000 7fc642ffd700 1 -- 172.21.15.87:0/157340932 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7fc64800f2e0 con 0x7fc64c107fa0 2024-03-21T08:48:06.410 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc638039220 msgr2=0x7fc63803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:06.410 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc638039220 0x7fc63803b6e0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc63c006fd0 tx=0x7fc63c006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:06.410 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 msgr2=0x7fc64c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c19bb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc6480116a0 tx=0x7fc648011ba0 comp rx=0 tx=0).stop 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 shutdown_connections 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc638039220 0x7fc63803b6e0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 --2- 172.21.15.87:0/157340932 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc64c107fa0 0x7fc64c19bb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 >> 172.21.15.87:0/157340932 conn(0x7fc64c101740 msgr2=0x7fc64c102620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 shutdown_connections 2024-03-21T08:48:06.411 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:05.684+0000 7fc653bf1700 1 -- 172.21.15.87:0/157340932 wait complete. 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.798+0000 7fc8b2fa7700 1 Processor -- start 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.798+0000 7fc8b2fa7700 1 -- start start 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b2fa7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac10a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b2fa7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac07be90 con 0x7fc8ac108530 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b0d43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac10a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b0d43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac10a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49332/0 (socket says 172.21.15.87:49332) 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b0d43700 1 -- 172.21.15.87:0/3618405199 learned_addr learned my addr 172.21.15.87:0/3618405199 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:07.680 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b0d43700 1 -- 172.21.15.87:0/3618405199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac10ae60 con 0x7fc8ac108530 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8b0d43700 1 --2- 172.21.15.87:0/3618405199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac10a920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00098a0 tx=0x7fc8a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb9e79885435d0b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8ab7fe700 1 -- 172.21.15.87:0/3618405199 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a0004160 con 0x7fc8ac108530 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8ab7fe700 1 -- 172.21.15.87:0/3618405199 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a0004300 con 0x7fc8ac108530 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.799+0000 7fc8ab7fe700 1 -- 172.21.15.87:0/3618405199 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a0011b20 con 0x7fc8ac108530 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/3618405199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 msgr2=0x7fc8ac10a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 --2- 172.21.15.87:0/3618405199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac10a920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00098a0 tx=0x7fc8a0009bb0 comp rx=0 tx=0).stop 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/3618405199 shutdown_connections 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 --2- 172.21.15.87:0/3618405199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac10a920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/3618405199 >> 172.21.15.87:0/3618405199 conn(0x7fc8ac101f60 msgr2=0x7fc8ac104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/3618405199 shutdown_connections 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.800+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/3618405199 wait complete. 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b2fa7700 1 Processor -- start 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b2fa7700 1 -- start start 2024-03-21T08:48:07.681 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b2fa7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac107db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b2fa7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ac10bec0 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b0d43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac107db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b0d43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac107db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49334/0 (socket says 172.21.15.87:49334) 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b0d43700 1 -- 172.21.15.87:0/1908035989 learned_addr learned my addr 172.21.15.87:0/1908035989 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b0d43700 1 -- 172.21.15.87:0/1908035989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ac1082f0 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b0d43700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac107db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00117e0 tx=0x7fc8a0011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a0003ba0 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8a001a910 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc8a00248f0 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.801+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ac1064e0 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.802+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ac1069a0 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.802+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fc8a0022020 con 0x7fc8ac108530 2024-03-21T08:48:07.682 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.803+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ac0637c0 con 0x7fc8ac108530 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.803+0000 7fc8a9ffb700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc89c039270 0x7fc89c03b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.803+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc8a004dc30 con 0x7fc8ac108530 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.803+0000 7fc8abfff700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc89c039270 0x7fc89c03b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.803+0000 7fc8abfff700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc89c039270 0x7fc89c03b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc898006fd0 tx=0x7fc898006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:06.808+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8a001ab70 con 0x7fc8ac108530 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.109+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fc8ac106c20 con 0x7fc8ac108530 2024-03-21T08:48:07.683 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.112+0000 7fc8a9ffb700 1 -- 172.21.15.87:0/1908035989 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fc8a0010380 con 0x7fc8ac108530 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc89c039270 msgr2=0x7fc89c03b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc89c039270 0x7fc89c03b730 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fc898006fd0 tx=0x7fc898006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 msgr2=0x7fc8ac107db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac107db0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00117e0 tx=0x7fc8a0011ba0 comp rx=0 tx=0).stop 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 shutdown_connections 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc89c039270 0x7fc89c03b730 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:07.684 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 --2- 172.21.15.87:0/1908035989 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8ac108530 0x7fc8ac107db0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:07.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 >> 172.21.15.87:0/1908035989 conn(0x7fc8ac101f60 msgr2=0x7fc8ac1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:07.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 shutdown_connections 2024-03-21T08:48:07.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr 2024-03-21T08:48:07.114+0000 7fc8b2fa7700 1 -- 172.21.15.87:0/1908035989 wait complete. 2024-03-21T08:48:07.686 INFO:teuthology.orchestra.run.smithi087.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-21T08:48:07.686 INFO:teuthology.orchestra.run.smithi087.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-21T08:48:07.686 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout:Or, if you are only running a single cluster on this host: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.687 INFO:teuthology.orchestra.run.smithi087.stdout: ceph telemetry on 2024-03-21T08:48:07.688 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.688 INFO:teuthology.orchestra.run.smithi087.stdout:For more information see: 2024-03-21T08:48:07.688 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.688 INFO:teuthology.orchestra.run.smithi087.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-21T08:48:07.688 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:07.688 INFO:teuthology.orchestra.run.smithi087.stdout:Bootstrap complete. 2024-03-21T08:48:07.723 INFO:tasks.cephadm:Fetching config... 2024-03-21T08:48:07.723 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:48:07.723 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-21T08:48:07.740 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-21T08:48:07.740 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:48:07.740 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-21T08:48:07.795 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-21T08:48:07.795 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:48:07.795 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-21T08:48:07.862 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-21T08:48:07.863 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:48:07.863 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-21T08:48:07.918 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-21T08:48:07.918 DEBUG:teuthology.orchestra.run.smithi087:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T08:48:07.997 INFO:teuthology.orchestra.run.smithi087.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:48:08.012 DEBUG:teuthology.orchestra.run.smithi100:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T08:48:08.057 INFO:teuthology.orchestra.run.smithi100.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:48:08.072 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-21T08:48:08.118 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrfgM1uysRgnuneUPwI0HFcdlZQgJGBUvrRVJ6gtwAVgDaV0XnxaFbs6UFV5/H8jySfmRLQJH6vZUGqJvgIQywZ67VwVH5LQiDnx0qdOoC7RvXLz+mCo40i9Itcqs27mX374JLJ6LBQAMg2njaJYHpEQBLNcBORDpEnKQmwW82oNhrKYZId1SGrl9kZco//TE5smS67lmFjBsXWRoMN9ULHTu5tuQA+uB6yo5xuAoQhX4w9rlx5CrLiV2nsUeHlqtr9LG1Dl8/tnJfx0rLb4pEtmZ3sfIIuPGQOhNeryp9s7BsL/SedgGZLB/Shcml1VnL18OP3fGECMQ3Ptjfo3jRVi+hcIwKUWh8qWzwhkZrEb22CJSTuqveiLfo1lGwXwf4k32g5uYHvQdoBTNZo2duLW+j1jJV3Vj3306S38GSfeet7fj/rmzdVubb0+9rNlDjL6bPwT8OZ16n/miCedQlAPKIougmDFxbEFYfiftieJ+uSxCW77lum8FXTCMSnl0= ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:48:08.133 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-21T08:48:08.241 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:08 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1908035989' entity='client.admin' 2024-03-21T08:48:08.383 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/4141267090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 msgr2=0x7f3a38073e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 --2- 172.21.15.87:0/4141267090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a38073e70 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3a28009910 tx=0x7f3a28009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/4141267090 shutdown_connections 2024-03-21T08:48:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 --2- 172.21.15.87:0/4141267090 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a38073e70 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/4141267090 >> 172.21.15.87:0/4141267090 conn(0x7f3a3806ea40 msgr2=0x7f3a38070e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:10.625 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/4141267090 shutdown_connections 2024-03-21T08:48:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.627+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/4141267090 wait complete. 2024-03-21T08:48:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a3d8e3700 1 Processor -- start 2024-03-21T08:48:10.626 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a3d8e3700 1 -- start start 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a3d8e3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a3819faa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a3d8e3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a38110460 con 0x7f3a38073a50 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a36ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a3819faa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a36ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a3819faa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34092/0 (socket says 172.21.15.87:34092) 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a36ffd700 1 -- 172.21.15.87:0/1745295142 learned_addr learned my addr 172.21.15.87:0/1745295142 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.628+0000 7f3a36ffd700 1 -- 172.21.15.87:0/1745295142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3819ffe0 con 0x7f3a38073a50 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.629+0000 7f3a36ffd700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a3819faa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a28004fa0 tx=0x7f3a28005080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.629+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a28027070 con 0x7f3a38073a50 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.629+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a2800be40 con 0x7f3a38073a50 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.629+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3a2800f460 con 0x7f3a38073a50 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.629+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a381a0270 con 0x7f3a38073a50 2024-03-21T08:48:10.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.629+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a381a0690 con 0x7f3a38073a50 2024-03-21T08:48:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.630+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f3a2802a070 con 0x7f3a38073a50 2024-03-21T08:48:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.630+0000 7f3a3c8e1700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a20039280 0x7f3a2003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.630+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3a2801e030 con 0x7f3a38073a50 2024-03-21T08:48:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.630+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a38110be0 con 0x7f3a38073a50 2024-03-21T08:48:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.630+0000 7f3a367fc700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a20039280 0x7f3a2003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:10.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.631+0000 7f3a367fc700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a20039280 0x7f3a2003b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c006fd0 tx=0x7f3a2c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:10.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.635+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a28013070 con 0x7f3a38073a50 2024-03-21T08:48:10.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.840+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f3a3802dbe0 con 0x7f3a38073a50 2024-03-21T08:48:10.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.843+0000 7f3a3c8e1700 1 -- 172.21.15.87:0/1745295142 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f3a280213d0 con 0x7f3a38073a50 2024-03-21T08:48:10.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a20039280 msgr2=0x7f3a2003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:10.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a20039280 0x7f3a2003b740 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3a2c006fd0 tx=0x7f3a2c006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:10.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 msgr2=0x7f3a3819faa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:10.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a3819faa0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3a28004fa0 tx=0x7f3a28005080 comp rx=0 tx=0).stop 2024-03-21T08:48:10.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 shutdown_connections 2024-03-21T08:48:10.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a20039280 0x7f3a2003b740 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:10.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 --2- 172.21.15.87:0/1745295142 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a38073a50 0x7f3a3819faa0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:10.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 >> 172.21.15.87:0/1745295142 conn(0x7f3a3806ea40 msgr2=0x7f3a3806f470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:10.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 shutdown_connections 2024-03-21T08:48:10.844 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:10.846+0000 7f3a3d8e3700 1 -- 172.21.15.87:0/1745295142 wait complete. 2024-03-21T08:48:11.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi087", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1745295142' entity='client.admin' 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:11.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:11.488 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-21T08:48:11.488 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-21T08:48:11.748 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:13.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 -- 172.21.15.87:0/3231524193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 msgr2=0x7f6af8100ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 --2- 172.21.15.87:0/3231524193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af8100ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009910 tx=0x7f6aec009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 -- 172.21.15.87:0/3231524193 shutdown_connections 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 --2- 172.21.15.87:0/3231524193 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af8100ef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 -- 172.21.15.87:0/3231524193 >> 172.21.15.87:0/3231524193 conn(0x7f6af80fc090 msgr2=0x7f6af80fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 -- 172.21.15.87:0/3231524193 shutdown_connections 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.993+0000 7f6afefa7700 1 -- 172.21.15.87:0/3231524193 wait complete. 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afefa7700 1 Processor -- start 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afefa7700 1 -- start start 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afefa7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afefa7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af8106b60 con 0x7f6af8100ad0 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afcd43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:13.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afcd43700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af806aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34106/0 (socket says 172.21.15.87:34106) 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.994+0000 7f6afcd43700 1 -- 172.21.15.87:0/1586123113 learned_addr learned my addr 172.21.15.87:0/1586123113 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6afcd43700 1 -- 172.21.15.87:0/1586123113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8199fd0 con 0x7f6af8100ad0 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6afcd43700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af806aad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6aec006010 tx=0x7f6aec005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6aec027070 con 0x7f6af8100ad0 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6aec00be40 con 0x7f6af8100ad0 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af819a1a0 con 0x7f6af8100ad0 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6aec00f460 con 0x7f6af8100ad0 2024-03-21T08:48:13.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.995+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af819a5c0 con 0x7f6af8100ad0 2024-03-21T08:48:13.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.996+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f6aec02a070 con 0x7f6af8100ad0 2024-03-21T08:48:13.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.996+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af81072e0 con 0x7f6af8100ad0 2024-03-21T08:48:13.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.996+0000 7f6af5ffb700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6ae8039280 0x7f6ae803b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:13.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.996+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6aec01e030 con 0x7f6af8100ad0 2024-03-21T08:48:13.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.997+0000 7f6af7fff700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6ae8039280 0x7f6ae803b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:13.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:13.997+0000 7f6af7fff700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6ae8039280 0x7f6ae803b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4006fd0 tx=0x7f6ae4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:13.998 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.001+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6aec013070 con 0x7f6af8100ad0 2024-03-21T08:48:14.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.203+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f6af802dc60 con 0x7f6ae8039280 2024-03-21T08:48:14.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.206+0000 7f6af5ffb700 1 -- 172.21.15.87:0/1586123113 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f6af802dc60 con 0x7f6ae8039280 2024-03-21T08:48:14.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6ae8039280 msgr2=0x7f6ae803b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:14.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6ae8039280 0x7f6ae803b740 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4006fd0 tx=0x7f6ae4006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:14.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 msgr2=0x7f6af806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:14.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af806aad0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6aec006010 tx=0x7f6aec005050 comp rx=0 tx=0).stop 2024-03-21T08:48:14.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 shutdown_connections 2024-03-21T08:48:14.206 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6ae8039280 0x7f6ae803b740 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:14.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 --2- 172.21.15.87:0/1586123113 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6af8100ad0 0x7f6af806aad0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:14.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 >> 172.21.15.87:0/1586123113 conn(0x7f6af80fc090 msgr2=0x7f6af80fcaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:14.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 shutdown_connections 2024-03-21T08:48:14.207 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:14.209+0000 7f6afefa7700 1 -- 172.21.15.87:0/1586123113 wait complete. 2024-03-21T08:48:14.978 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi100 2024-03-21T08:48:14.978 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:48:14.978 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.conf 2024-03-21T08:48:14.997 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:48:14.997 DEBUG:teuthology.orchestra.run.smithi100:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:48:15.055 INFO:tasks.cephadm:Adding host smithi100 to orchestrator... 2024-03-21T08:48:15.055 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch host add smithi100 2024-03-21T08:48:15.293 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:15.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:48:15.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:15.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:15.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:15.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:48:15.530 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: Updating smithi087:/etc/ceph/ceph.conf 2024-03-21T08:48:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: Updating smithi087:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:48:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: Updating smithi087:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:48:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: Updating smithi087:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:48:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 -- 172.21.15.87:0/1864161234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 msgr2=0x7f220c106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 --2- 172.21.15.87:0/1864161234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c106e40 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f21fc009910 tx=0x7f21fc009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:16.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 -- 172.21.15.87:0/1864161234 shutdown_connections 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 --2- 172.21.15.87:0/1864161234 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c106e40 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 -- 172.21.15.87:0/1864161234 >> 172.21.15.87:0/1864161234 conn(0x7f220c0776d0 msgr2=0x7f220c0fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 -- 172.21.15.87:0/1864161234 shutdown_connections 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 -- 172.21.15.87:0/1864161234 wait complete. 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.680+0000 7f221130d700 1 Processor -- start 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f221130d700 1 -- start start 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f221130d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c1976f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f221130d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f220c10b0a0 con 0x7f220c106a60 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f220affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c1976f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f220affd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c1976f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34134/0 (socket says 172.21.15.87:34134) 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f220affd700 1 -- 172.21.15.87:0/1086585716 learned_addr learned my addr 172.21.15.87:0/1086585716 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f220affd700 1 -- 172.21.15.87:0/1086585716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c197c30 con 0x7f220c106a60 2024-03-21T08:48:16.679 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f220affd700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c1976f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006010 tx=0x7f21fc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.681+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21fc026070 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.682+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f220c197f20 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.682+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f21fc00bbd0 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.682+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f220c19bfc0 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.682+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f21fc003910 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.683+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f21fc029070 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.683+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220c1021a0 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.683+0000 7f2208ff9700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f21f8039220 0x7f21f803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.683+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f21fc01e070 con 0x7f220c106a60 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.683+0000 7f22027fc700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f21f8039220 0x7f21f803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:16.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.683+0000 7f22027fc700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f21f8039220 0x7f21f803b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21f4006fd0 tx=0x7f21f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:16.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.688+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f21fc013070 con 0x7f220c106a60 2024-03-21T08:48:16.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:16.887+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}) v1 -- 0x7f220c198a50 con 0x7f21f8039220 2024-03-21T08:48:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:17 smithi087 ceph-mon[32385]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi100", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:48:18.310 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:18 smithi087 ceph-mon[32385]: Deploying cephadm binary to smithi100 2024-03-21T08:48:19.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:19 smithi087 ceph-mon[32385]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:21.074 INFO:teuthology.orchestra.run.smithi087.stdout:Added host 'smithi100' with addr '172.21.15.100' 2024-03-21T08:48:21.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.077+0000 7f2208ff9700 1 -- 172.21.15.87:0/1086585716 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f220c198a50 con 0x7f21f8039220 2024-03-21T08:48:21.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f21f8039220 msgr2=0x7f21f803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f21f8039220 0x7f21f803b6e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f21f4006fd0 tx=0x7f21f4006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 msgr2=0x7f220c1976f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c1976f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f21fc006010 tx=0x7f21fc005030 comp rx=0 tx=0).stop 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 shutdown_connections 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f21f8039220 0x7f21f803b6e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 --2- 172.21.15.87:0/1086585716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f220c106a60 0x7f220c1976f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 >> 172.21.15.87:0/1086585716 conn(0x7f220c0776d0 msgr2=0x7f220c069580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 shutdown_connections 2024-03-21T08:48:21.077 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:21.079+0000 7f221130d700 1 -- 172.21.15.87:0/1086585716 wait complete. 2024-03-21T08:48:21.688 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T08:48:21.921 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:22.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:21 smithi087 ceph-mon[32385]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:22.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:22.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:22.044 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:23.018 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:22 smithi087 ceph-mon[32385]: Added host smithi100 2024-03-21T08:48:23.018 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:22 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:23.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 -- 172.21.15.87:0/329957771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 msgr2=0x7fcf40074910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:23.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 --2- 172.21.15.87:0/329957771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf40074910 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcf34009910 tx=0x7fcf34009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:23.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 -- 172.21.15.87:0/329957771 shutdown_connections 2024-03-21T08:48:23.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 --2- 172.21.15.87:0/329957771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf40074910 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:23.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 -- 172.21.15.87:0/329957771 >> 172.21.15.87:0/329957771 conn(0x7fcf400fd6a0 msgr2=0x7fcf400ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:23.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 -- 172.21.15.87:0/329957771 shutdown_connections 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.515+0000 7fcf45e20700 1 -- 172.21.15.87:0/329957771 wait complete. 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf45e20700 1 Processor -- start 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf45e20700 1 -- start start 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf45e20700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf4019dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf45e20700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40112810 con 0x7fcf40074530 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf3f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf4019dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf3f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf4019dd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49894/0 (socket says 172.21.15.87:49894) 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.516+0000 7fcf3f7fe700 1 -- 172.21.15.87:0/2242704798 learned_addr learned my addr 172.21.15.87:0/2242704798 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf3f7fe700 1 -- 172.21.15.87:0/2242704798 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4019e2b0 con 0x7fcf40074530 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf3f7fe700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf4019dd70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf34006010 tx=0x7fcf34005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf34026070 con 0x7fcf40074530 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcf3400bc30 con 0x7fcf40074530 2024-03-21T08:48:23.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4019e540 con 0x7fcf40074530 2024-03-21T08:48:23.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcf34003ab0 con 0x7fcf40074530 2024-03-21T08:48:23.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.517+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4019e9a0 con 0x7fcf40074530 2024-03-21T08:48:23.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.518+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fcf34029070 con 0x7fcf40074530 2024-03-21T08:48:23.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.518+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf40075c70 con 0x7fcf40074530 2024-03-21T08:48:23.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.518+0000 7fcf3cff9700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcf28039220 0x7fcf2803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:23.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.518+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcf3401e070 con 0x7fcf40074530 2024-03-21T08:48:23.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.518+0000 7fcf3effd700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcf28039220 0x7fcf2803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:23.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.518+0000 7fcf3effd700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcf28039220 0x7fcf2803b6e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf30006fd0 tx=0x7fcf30006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:23.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.523+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcf34013070 con 0x7fcf40074530 2024-03-21T08:48:23.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.721+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf4019f0d0 con 0x7fcf28039220 2024-03-21T08:48:23.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.722+0000 7fcf3cff9700 1 -- 172.21.15.87:0/2242704798 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7fcf4019f0d0 con 0x7fcf28039220 2024-03-21T08:48:23.720 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:23.720 INFO:teuthology.orchestra.run.smithi087.stdout:[{"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}] 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcf28039220 msgr2=0x7fcf2803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcf28039220 0x7fcf2803b6e0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fcf30006fd0 tx=0x7fcf30006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 msgr2=0x7fcf4019dd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf4019dd70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fcf34006010 tx=0x7fcf34005050 comp rx=0 tx=0).stop 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 shutdown_connections 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcf28039220 0x7fcf2803b6e0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 --2- 172.21.15.87:0/2242704798 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcf40074530 0x7fcf4019dd70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.724+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 >> 172.21.15.87:0/2242704798 conn(0x7fcf400fd6a0 msgr2=0x7fcf400ff0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.725+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 shutdown_connections 2024-03-21T08:48:23.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:23.725+0000 7fcf45e20700 1 -- 172.21.15.87:0/2242704798 wait complete. 2024-03-21T08:48:24.065 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:23 smithi087 ceph-mon[32385]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:24.065 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:24.368 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi105 2024-03-21T08:48:24.369 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:48:24.369 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.conf 2024-03-21T08:48:24.389 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:48:24.389 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:48:24.447 INFO:tasks.cephadm:Adding host smithi105 to orchestrator... 2024-03-21T08:48:24.447 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch host add smithi105 2024-03-21T08:48:24.678 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:25.103 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:24 smithi087 ceph-mon[32385]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:48:25.960 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:25 smithi087 ceph-mon[32385]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:26.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/4055659051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 msgr2=0x7fa29c1021b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:26.191 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 --2- 172.21.15.87:0/4055659051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c1021b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fa29400b0a0 tx=0x7fa29400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/4055659051 shutdown_connections 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 --2- 172.21.15.87:0/4055659051 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c1021b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/4055659051 >> 172.21.15.87:0/4055659051 conn(0x7fa29c0fd6a0 msgr2=0x7fa29c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/4055659051 shutdown_connections 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.191+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/4055659051 wait complete. 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a2ed4700 1 Processor -- start 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a2ed4700 1 -- start start 2024-03-21T08:48:26.192 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a2ed4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a2ed4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa29c10c1a0 con 0x7fa29c101dd0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a0c70700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a0c70700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49906/0 (socket says 172.21.15.87:49906) 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a0c70700 1 -- 172.21.15.87:0/3553699841 learned_addr learned my addr 172.21.15.87:0/3553699841 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.192+0000 7fa2a0c70700 1 -- 172.21.15.87:0/3553699841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c076380 con 0x7fa29c101dd0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.193+0000 7fa2a0c70700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c074490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa294015040 tx=0x7fa294003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.193+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa294017040 con 0x7fa29c101dd0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.193+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa294022070 con 0x7fa29c101dd0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.193+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa29c074a30 con 0x7fa29c101dd0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.193+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa29401d5d0 con 0x7fa29c101dd0 2024-03-21T08:48:26.193 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.193+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29c074ef0 con 0x7fa29c101dd0 2024-03-21T08:48:26.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.194+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fa29401d8d0 con 0x7fa29c101dd0 2024-03-21T08:48:26.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.194+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa29c1032b0 con 0x7fa29c101dd0 2024-03-21T08:48:26.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.194+0000 7fa299ffb700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa288039220 0x7fa28803b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:26.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.194+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa29400f050 con 0x7fa29c101dd0 2024-03-21T08:48:26.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.194+0000 7fa29bfff700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa288039220 0x7fa28803b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:26.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.195+0000 7fa29bfff700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa288039220 0x7fa28803b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa284006fd0 tx=0x7fa284006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:26.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.201+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa294007e20 con 0x7fa29c101dd0 2024-03-21T08:48:26.402 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:26.404+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}) v1 -- 0x7fa29c075a30 con 0x7fa288039220 2024-03-21T08:48:27.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:27.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:27.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi100", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: Updating smithi100:/etc/ceph/ceph.conf 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: Updating smithi100:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: Updating smithi100:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: Updating smithi100:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:27.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:27.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:28.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:28 smithi087 ceph-mon[32385]: Deploying cephadm binary to smithi105 2024-03-21T08:48:29.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:29 smithi087 ceph-mon[32385]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:30.668 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.669+0000 7fa299ffb700 1 -- 172.21.15.87:0/3553699841 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa29c075a30 con 0x7fa288039220 2024-03-21T08:48:30.668 INFO:teuthology.orchestra.run.smithi087.stdout:Added host 'smithi105' with addr '172.21.15.105' 2024-03-21T08:48:30.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa288039220 msgr2=0x7fa28803b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:30.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa288039220 0x7fa28803b6e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa284006fd0 tx=0x7fa284006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:30.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 msgr2=0x7fa29c074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:30.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c074490 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa294015040 tx=0x7fa294003d50 comp rx=0 tx=0).stop 2024-03-21T08:48:30.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 shutdown_connections 2024-03-21T08:48:30.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa288039220 0x7fa28803b6e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:30.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 --2- 172.21.15.87:0/3553699841 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa29c101dd0 0x7fa29c074490 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:30.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 >> 172.21.15.87:0/3553699841 conn(0x7fa29c0fd6a0 msgr2=0x7fa29c1037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:30.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 shutdown_connections 2024-03-21T08:48:30.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:30.672+0000 7fa2a2ed4700 1 -- 172.21.15.87:0/3553699841 wait complete. 2024-03-21T08:48:31.412 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch host ls --format=json 2024-03-21T08:48:31.651 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:32.006 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:32.007 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:31 smithi087 ceph-mon[32385]: Added host smithi105 2024-03-21T08:48:32.007 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:32.007 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:31 smithi087 ceph-mon[32385]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:32.007 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:32.711 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 -- 172.21.15.87:0/639433991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 msgr2=0x7fb4f8103100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 --2- 172.21.15.87:0/639433991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8103100 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8009910 tx=0x7fb4e8009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 -- 172.21.15.87:0/639433991 shutdown_connections 2024-03-21T08:48:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 --2- 172.21.15.87:0/639433991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8103100 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 -- 172.21.15.87:0/639433991 >> 172.21.15.87:0/639433991 conn(0x7fb4f80776d0 msgr2=0x7fb4f80fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 -- 172.21.15.87:0/639433991 shutdown_connections 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 -- 172.21.15.87:0/639433991 wait complete. 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.172+0000 7fb4fcac8700 1 Processor -- start 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4fcac8700 1 -- start start 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4fcac8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8197740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4fcac8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f810c210 con 0x7fb4f8102d20 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8197740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4f659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8197740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45290/0 (socket says 172.21.15.87:45290) 2024-03-21T08:48:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4f659c700 1 -- 172.21.15.87:0/3078990241 learned_addr learned my addr 172.21.15.87:0/3078990241 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4f659c700 1 -- 172.21.15.87:0/3078990241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f8197c80 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.173+0000 7fb4f659c700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8197740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8006010 tx=0x7fb4e8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.174+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4e8026070 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.174+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4e800bc30 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.174+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f8197f10 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.174+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb4e80039a0 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.174+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f8198370 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.175+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fb4e8029070 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.175+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f8069e60 con 0x7fb4f8102d20 2024-03-21T08:48:33.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.175+0000 7fb4effff700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e4039280 0x7fb4e403b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:33.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.175+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb4e801e070 con 0x7fb4f8102d20 2024-03-21T08:48:33.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.175+0000 7fb4edd9b700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e4039280 0x7fb4e403b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:33.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.175+0000 7fb4edd9b700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e4039280 0x7fb4e403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006fd0 tx=0x7fb4e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:33.178 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.180+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4e8013070 con 0x7fb4f8102d20 2024-03-21T08:48:33.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.379+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4f8198aa0 con 0x7fb4e4039280 2024-03-21T08:48:33.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.380+0000 7fb4effff700 1 -- 172.21.15.87:0/3078990241 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+240 (secure 0 0 0) 0x7fb4f8198aa0 con 0x7fb4e4039280 2024-03-21T08:48:33.378 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:48:33.379 INFO:teuthology.orchestra.run.smithi087.stdout:[{"addr": "172.21.15.87", "hostname": "smithi087", "labels": [], "status": ""}, {"addr": "172.21.15.100", "hostname": "smithi100", "labels": [], "status": ""}, {"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}] 2024-03-21T08:48:33.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e4039280 msgr2=0x7fb4e403b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:33.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e4039280 0x7fb4e403b740 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb4e0006fd0 tx=0x7fb4e0006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:33.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 msgr2=0x7fb4f8197740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:33.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8197740 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fb4e8006010 tx=0x7fb4e8005030 comp rx=0 tx=0).stop 2024-03-21T08:48:33.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 shutdown_connections 2024-03-21T08:48:33.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e4039280 0x7fb4e403b740 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 --2- 172.21.15.87:0/3078990241 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f8102d20 0x7fb4f8197740 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.382+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 >> 172.21.15.87:0/3078990241 conn(0x7fb4f80776d0 msgr2=0x7fb4f8103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.383+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 shutdown_connections 2024-03-21T08:48:33.382 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:33.383+0000 7fb4fcac8700 1 -- 172.21.15.87:0/3078990241 wait complete. 2024-03-21T08:48:33.958 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:33.958 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:33 smithi087 ceph-mon[32385]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:33.986 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-21T08:48:33.986 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd crush tunables default 2024-03-21T08:48:34.222 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:48:34.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:34 smithi087 ceph-mon[32385]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:48:35.724 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 -- 172.21.15.87:0/558187583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 msgr2=0x7f42d4100f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 --2- 172.21.15.87:0/558187583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d4100f00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f42d0009910 tx=0x7f42d0009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 -- 172.21.15.87:0/558187583 shutdown_connections 2024-03-21T08:48:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 --2- 172.21.15.87:0/558187583 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d4100f00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 -- 172.21.15.87:0/558187583 >> 172.21.15.87:0/558187583 conn(0x7f42d40fc580 msgr2=0x7f42d40fe9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:35.725 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 -- 172.21.15.87:0/558187583 shutdown_connections 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 -- 172.21.15.87:0/558187583 wait complete. 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 Processor -- start 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.724+0000 7f42dc21e700 1 -- start start 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42dc21e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d41a3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42dc21e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d4112820 con 0x7f42d4100b20 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42d9fba700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d41a3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42d9fba700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d41a3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45322/0 (socket says 172.21.15.87:45322) 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42d9fba700 1 -- 172.21.15.87:0/1974626985 learned_addr learned my addr 172.21.15.87:0/1974626985 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42d9fba700 1 -- 172.21.15.87:0/1974626985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d41a3f80 con 0x7f42d4100b20 2024-03-21T08:48:35.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.725+0000 7f42d9fba700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d41a3a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006010 tx=0x7f42d0005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:35.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.726+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42d0026070 con 0x7f42d4100b20 2024-03-21T08:48:35.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.726+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d41a4210 con 0x7f42d4100b20 2024-03-21T08:48:35.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.726+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42d000bc30 con 0x7f42d4100b20 2024-03-21T08:48:35.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.726+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f42d00039a0 con 0x7f42d4100b20 2024-03-21T08:48:35.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.726+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d419db10 con 0x7f42d4100b20 2024-03-21T08:48:35.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.727+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f42d0029070 con 0x7f42d4100b20 2024-03-21T08:48:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.727+0000 7f42caffd700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42c0039280 0x7f42c003b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.727+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f42d001e070 con 0x7f42d4100b20 2024-03-21T08:48:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.727+0000 7f42d97b9700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42c0039280 0x7f42c003b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.727+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d406a8d0 con 0x7f42d4100b20 2024-03-21T08:48:35.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.728+0000 7f42d97b9700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42c0039280 0x7f42c003b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42c4006fd0 tx=0x7f42c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:35.732 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.734+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f42d0030020 con 0x7f42d4100b20 2024-03-21T08:48:35.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:35.937+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f42d40637c0 con 0x7f42d4100b20 2024-03-21T08:48:36.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:35 smithi087 ceph-mon[32385]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:36.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.901+0000 7f42caffd700 1 -- 172.21.15.87:0/1974626985 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f42d0013070 con 0x7f42d4100b20 2024-03-21T08:48:36.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42c0039280 msgr2=0x7f42c003b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42c0039280 0x7f42c003b740 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f42c4006fd0 tx=0x7f42c4006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 msgr2=0x7f42d41a3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d41a3a40 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f42d0006010 tx=0x7f42d0005030 comp rx=0 tx=0).stop 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 shutdown_connections 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42c0039280 0x7f42c003b740 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 --2- 172.21.15.87:0/1974626985 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42d4100b20 0x7f42d41a3a40 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 >> 172.21.15.87:0/1974626985 conn(0x7f42d40fc580 msgr2=0x7f42d40fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 shutdown_connections 2024-03-21T08:48:36.902 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:48:36.903+0000 7f42dc21e700 1 -- 172.21.15.87:0/1974626985 wait complete. 2024-03-21T08:48:36.903 INFO:teuthology.orchestra.run.smithi087.stderr:adjusted tunables profile to default 2024-03-21T08:48:37.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: Updating smithi105:/etc/ceph/ceph.conf 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1974626985' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:37.447 INFO:tasks.cephadm:Adding mon.a on smithi087 2024-03-21T08:48:37.447 INFO:tasks.cephadm:Adding mon.b on smithi100 2024-03-21T08:48:37.447 INFO:tasks.cephadm:Adding mon.c on smithi105 2024-03-21T08:48:37.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch apply mon '3;smithi087:172.21.15.87=a;smithi100:172.21.15.100=b;smithi105:172.21.15.105=c' 2024-03-21T08:48:37.673 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:48:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:37 smithi087 ceph-mon[32385]: Updating smithi105:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.conf 2024-03-21T08:48:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:37 smithi087 ceph-mon[32385]: Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:48:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:37 smithi087 ceph-mon[32385]: Updating smithi105:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/config/ceph.client.admin.keyring 2024-03-21T08:48:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:37 smithi087 ceph-mon[32385]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:38.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:37 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1974626985' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-21T08:48:38.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:37 smithi087 ceph-mon[32385]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:48:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 -- 172.21.15.105:0/1406973574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 msgr2=0x7f1208100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 --2- 172.21.15.105:0/1406973574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f1208100790 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009910 tx=0x7f11f8009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 -- 172.21.15.105:0/1406973574 shutdown_connections 2024-03-21T08:48:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 --2- 172.21.15.105:0/1406973574 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f1208100790 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 -- 172.21.15.105:0/1406973574 >> 172.21.15.105:0/1406973574 conn(0x7f1208077140 msgr2=0x7f1208077550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:39.108 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 -- 172.21.15.105:0/1406973574 shutdown_connections 2024-03-21T08:48:39.109 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.111+0000 7f120ced8700 1 -- 172.21.15.105:0/1406973574 wait complete. 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120ced8700 1 Processor -- start 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120ced8700 1 -- start start 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120ced8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f12081976a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120ced8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120810c240 con 0x7f12081003b0 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f12081976a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f12081976a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.105:45706/0 (socket says 172.21.15.105:45706) 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120659c700 1 -- 172.21.15.105:0/2937945738 learned_addr learned my addr 172.21.15.105:0/2937945738 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.112+0000 7f120659c700 1 -- 172.21.15.105:0/2937945738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208197be0 con 0x7f12081003b0 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.113+0000 7f120659c700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f12081976a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005850 tx=0x7f11f8005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.113+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11f8026070 con 0x7f12081003b0 2024-03-21T08:48:39.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.113+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1208197ed0 con 0x7f12081003b0 2024-03-21T08:48:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.113+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f11f800bbd0 con 0x7f12081003b0 2024-03-21T08:48:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.113+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f11f8003970 con 0x7f12081003b0 2024-03-21T08:48:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.113+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120819bf70 con 0x7f12081003b0 2024-03-21T08:48:39.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.114+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12080637c0 con 0x7f12081003b0 2024-03-21T08:48:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.114+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f11f8029070 con 0x7f12081003b0 2024-03-21T08:48:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.114+0000 7f11ff7fe700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f11f4039220 0x7f11f403b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.114+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f11f801e070 con 0x7f12081003b0 2024-03-21T08:48:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.114+0000 7f11fffff700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f11f4039220 0x7f11f403b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:39.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.115+0000 7f11fffff700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f11f4039220 0x7f11f403b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11f0006fd0 tx=0x7f11f0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:39.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.119+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f11f8013070 con 0x7f12081003b0 2024-03-21T08:48:39.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.326+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi100:172.21.15.100=b;smithi105:172.21.15.105=c", "target": ["mon-mgr", ""]}) v1 -- 0x7f12080feae0 con 0x7f11f4039220 2024-03-21T08:48:39.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.333+0000 7f11ff7fe700 1 -- 172.21.15.105:0/2937945738 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f12080feae0 con 0x7f11f4039220 2024-03-21T08:48:39.330 INFO:teuthology.orchestra.run.smithi105.stdout:Scheduled mon update... 2024-03-21T08:48:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f11f4039220 msgr2=0x7f11f403b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f11f4039220 0x7f11f403b6e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f11f0006fd0 tx=0x7f11f0006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 msgr2=0x7f12081976a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f12081976a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f11f8005850 tx=0x7f11f8005030 comp rx=0 tx=0).stop 2024-03-21T08:48:39.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 shutdown_connections 2024-03-21T08:48:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f11f4039220 0x7f11f403b6e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 --2- 172.21.15.105:0/2937945738 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12081003b0 0x7f12081976a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 >> 172.21.15.105:0/2937945738 conn(0x7f1208077140 msgr2=0x7f12080fe3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 shutdown_connections 2024-03-21T08:48:39.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:39.335+0000 7f120ced8700 1 -- 172.21.15.105:0/2937945738 wait complete. 2024-03-21T08:48:39.930 DEBUG:teuthology.orchestra.run.smithi100:mon.b> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.b.service 2024-03-21T08:48:39.932 DEBUG:teuthology.orchestra.run.smithi105:mon.c> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.c.service 2024-03-21T08:48:39.935 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T08:48:39.935 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T08:48:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:48:40.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:41.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:40 smithi087 ceph-mon[32385]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi087:172.21.15.87=a;smithi100:172.21.15.100=b;smithi105:172.21.15.105=c", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:48:41.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:40 smithi087 ceph-mon[32385]: Saving service mon spec with placement smithi087:172.21.15.87=a;smithi100:172.21.15.100=b;smithi105:172.21.15.105=c;count:3 2024-03-21T08:48:41.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:40 smithi087 ceph-mon[32385]: Deploying daemon mon.c on smithi105 2024-03-21T08:48:41.947 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:48:42.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:41 smithi087 ceph-mon[32385]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 -- 172.21.15.105:0/1708456154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d20100df0 msgr2=0x7f6d201011d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 --2- 172.21.15.105:0/1708456154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d20100df0 0x7f6d201011d0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c009910 tx=0x7f6d1c009c20 comp rx=0 tx=0).stop 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 -- 172.21.15.105:0/1708456154 shutdown_connections 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 --2- 172.21.15.105:0/1708456154 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d20100df0 0x7f6d201011d0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 -- 172.21.15.105:0/1708456154 >> 172.21.15.105:0/1708456154 conn(0x7f6d200fc6a0 msgr2=0x7f6d200feac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 -- 172.21.15.105:0/1708456154 shutdown_connections 2024-03-21T08:48:43.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.863+0000 7f6d27f99700 1 -- 172.21.15.105:0/1708456154 wait complete. 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d27f99700 1 Processor -- start 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d27f99700 1 -- start start 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d27f99700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 0x7f6d2019dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d27f99700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2010b1c0 con 0x7f6d2019d6d0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d25d35700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 0x7f6d2019dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d25d35700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 0x7f6d2019dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.105:45732/0 (socket says 172.21.15.105:45732) 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d25d35700 1 -- 172.21.15.105:0/832362015 learned_addr learned my addr 172.21.15.105:0/832362015 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.864+0000 7f6d25d35700 1 -- 172.21.15.105:0/832362015 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d2019dff0 con 0x7f6d2019d6d0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.865+0000 7f6d25d35700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 0x7f6d2019dab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b5c0 tx=0x7f6d1c005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.865+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d1c026070 con 0x7f6d2019d6d0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.865+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d20197850 con 0x7f6d2019d6d0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.865+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d1c00b6f0 con 0x7f6d2019d6d0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.865+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d1c0037f0 con 0x7f6d2019d6d0 2024-03-21T08:48:43.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.865+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d20197d10 con 0x7f6d2019d6d0 2024-03-21T08:48:43.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.866+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f6d1c029070 con 0x7f6d2019d6d0 2024-03-21T08:48:43.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.866+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2019e280 con 0x7f6d2019d6d0 2024-03-21T08:48:43.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.866+0000 7f6d16ffd700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d0c039280 0x7f6d0c03b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:43.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.866+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6d1c01e070 con 0x7f6d2019d6d0 2024-03-21T08:48:43.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.866+0000 7f6d25534700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d0c039280 0x7f6d0c03b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:43.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.867+0000 7f6d25534700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d0c039280 0x7f6d0c03b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006fd0 tx=0x7f6d10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:43.869 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.872+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d1c013070 con 0x7f6d2019d6d0 2024-03-21T08:48:43.995 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:43.997+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6d1c02a3b0 con 0x7f6d2019d6d0 2024-03-21T08:48:44.150 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:44.152+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6d200637c0 con 0x7f6d2019d6d0 2024-03-21T08:48:44.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:43 smithi087 ceph-mon[32385]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:49.047 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:48:49.047 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","modified":"2024-03-21T08:48:43.995224Z","created":"2024-03-21T08:47:01.466296Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T08:48:49.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.049+0000 7f6d16ffd700 1 -- 172.21.15.105:0/832362015 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7f6d1c030070 con 0x7f6d2019d6d0 2024-03-21T08:48:49.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d0c039280 msgr2=0x7f6d0c03b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d0c039280 0x7f6d0c03b740 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f6d10006fd0 tx=0x7f6d10006e40 comp rx=0 tx=0).stop 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 msgr2=0x7f6d2019dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 0x7f6d2019dab0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00b5c0 tx=0x7f6d1c005030 comp rx=0 tx=0).stop 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 shutdown_connections 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d0c039280 0x7f6d0c03b740 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 --2- 172.21.15.105:0/832362015 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d2019d6d0 0x7f6d2019dab0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.051+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 >> 172.21.15.105:0/832362015 conn(0x7f6d200fc6a0 msgr2=0x7f6d200fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.052+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 shutdown_connections 2024-03-21T08:48:49.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:49.052+0000 7f6d27f99700 1 -- 172.21.15.105:0/832362015 wait complete. 2024-03-21T08:48:49.051 INFO:teuthology.orchestra.run.smithi105.stderr:dumped monmap epoch 2 2024-03-21T08:48:49.364 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: mon.a calling monitor election 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/832362015' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:49.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: mon.c calling monitor election 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: monmap e2: 2 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],c=[v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: fsmap 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: mgrmap e13: x(active, since 50s) 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: overall HEALTH_OK 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:49.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:49.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:49.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:48:49.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:48:50.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:50 smithi087 ceph-mon[32385]: Deploying daemon mon.b on smithi100 2024-03-21T08:48:50.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:50 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:50.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:50 smithi105 ceph-mon[35993]: Deploying daemon mon.b on smithi100 2024-03-21T08:48:50.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:50 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:50.681 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T08:48:50.682 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T08:48:50.925 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:48:51.300 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:51 smithi105 ceph-mon[35993]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:51 smithi087 ceph-mon[32385]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 -- 172.21.15.105:0/1572028373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b01005a0 msgr2=0x7fd0b0100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 --2- 172.21.15.105:0/1572028373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b01005a0 0x7fd0b0100980 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000aa20 tx=0x7fd0a000ad30 comp rx=0 tx=0).stop 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 -- 172.21.15.105:0/1572028373 shutdown_connections 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 --2- 172.21.15.105:0/1572028373 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b01005a0 0x7fd0b0100980 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 -- 172.21.15.105:0/1572028373 >> 172.21.15.105:0/1572028373 conn(0x7fd0b0077280 msgr2=0x7fd0b0077690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 -- 172.21.15.105:0/1572028373 shutdown_connections 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.358+0000 7fd0b499f700 1 -- 172.21.15.105:0/1572028373 wait complete. 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.359+0000 7fd0b499f700 1 Processor -- start 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.359+0000 7fd0b499f700 1 -- start start 2024-03-21T08:48:52.359 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.359+0000 7fd0b499f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.359+0000 7fd0b499f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b019fcf0 0x7fd0b01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.359+0000 7fd0b499f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b010a0a0 con 0x7fd0b019fcf0 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.359+0000 7fd0b499f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0b0109f20 con 0x7fd0b01005a0 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:55546/0 (socket says 172.21.15.105:55546) 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 -- 172.21.15.105:0/3130021165 learned_addr learned my addr 172.21.15.105:0/3130021165 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a7fff700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b019fcf0 0x7fd0b01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 -- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 msgr2=0x7fd0b019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 -- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 msgr2=0x7fd0b019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0ae59c700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a7fff700 1 -- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 msgr2=0x7fd0b019f7b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a7fff700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:52.360 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a7fff700 1 -- 172.21.15.105:0/3130021165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b01a0330 con 0x7fd0b019fcf0 2024-03-21T08:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a7fff700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b019fcf0 0x7fd0b01999c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b4d0 tx=0x7fd09800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd098011630 con 0x7fd0b019fcf0 2024-03-21T08:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0b0199f00 con 0x7fd0b019fcf0 2024-03-21T08:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd098011cb0 con 0x7fd0b019fcf0 2024-03-21T08:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fd098010e60 con 0x7fd0b019fcf0 2024-03-21T08:48:52.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.360+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0b019a490 con 0x7fd0b019fcf0 2024-03-21T08:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.361+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0b0101a30 con 0x7fd0b019fcf0 2024-03-21T08:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.361+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fd098018020 con 0x7fd0b019fcf0 2024-03-21T08:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.361+0000 7fd0a77fe700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd09c038f60 0x7fd09c03b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.362+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd09804bd80 con 0x7fd0b019fcf0 2024-03-21T08:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.362+0000 7fd0ae59c700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd09c038f60 0x7fd09c03b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:52.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.362+0000 7fd0ae59c700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd09c038f60 0x7fd09c03b420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000afd0 tx=0x7fd0a001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:52.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.367+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd09801f020 con 0x7fd0b019fcf0 2024-03-21T08:48:52.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.639+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd0b019af40 con 0x7fd0b019fcf0 2024-03-21T08:48:52.637 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.639+0000 7fd0a77fe700 1 -- 172.21.15.105:0/3130021165 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1015 (secure 0 0 0) 0x7fd09800fe40 con 0x7fd0b019fcf0 2024-03-21T08:48:52.638 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:48:52.638 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":2,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","modified":"2024-03-21T08:48:43.995224Z","created":"2024-03-21T08:47:01.466296Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T08:48:52.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd09c038f60 msgr2=0x7fd09c03b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:52.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd09c038f60 0x7fd09c03b420 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fd0a000afd0 tx=0x7fd0a001a040 comp rx=0 tx=0).stop 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b019fcf0 msgr2=0x7fd0b01999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b019fcf0 0x7fd0b01999c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fd09800b4d0 tx=0x7fd09800b890 comp rx=0 tx=0).stop 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 shutdown_connections 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd09c038f60 0x7fd09c03b420 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd0b01005a0 0x7fd0b019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 --2- 172.21.15.105:0/3130021165 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd0b019fcf0 0x7fd0b01999c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.641+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 >> 172.21.15.105:0/3130021165 conn(0x7fd0b0077280 msgr2=0x7fd0b0106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.642+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 shutdown_connections 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:52.642+0000 7fd0b499f700 1 -- 172.21.15.105:0/3130021165 wait complete. 2024-03-21T08:48:52.640 INFO:teuthology.orchestra.run.smithi105.stderr:dumped monmap epoch 2 2024-03-21T08:48:52.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:52 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/3130021165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:48:53.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:52 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/3130021165' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-21T08:48:54.246 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-21T08:48:54.246 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph mon dump -f json 2024-03-21T08:48:54.486 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:48:58.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:48:58.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: mon.c calling monitor election 2024-03-21T08:48:58.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: mon.a calling monitor election 2024-03-21T08:48:58.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: fsmap 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:48:58.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: mgrmap e13: x(active, since 59s) 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: mon.c calling monitor election 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: mon.a calling monitor election 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:58.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: fsmap 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: mgrmap e13: x(active, since 59s) 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: Health check failed: 1/3 mons down, quorum a,c (MON_DOWN) 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T08:48:58.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: [WRN] MON_DOWN: 1/3 mons down, quorum a,c 2024-03-21T08:48:58.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: mon.b (rank 2) addr [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] is down (out of quorum) 2024-03-21T08:48:58.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 -- 172.21.15.105:0/1642442944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 msgr2=0x7fc504005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 --2- 172.21.15.105:0/1642442944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc504005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc51c101b90 tx=0x7fc51000feb0 comp rx=0 tx=0).stop 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 -- 172.21.15.105:0/1642442944 shutdown_connections 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 --2- 172.21.15.105:0/1642442944 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc504005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 -- 172.21.15.105:0/1642442944 >> 172.21.15.105:0/1642442944 conn(0x7fc51c0fd730 msgr2=0x7fc51c0ffb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 -- 172.21.15.105:0/1642442944 shutdown_connections 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.947+0000 7fc5223b8700 1 -- 172.21.15.105:0/1642442944 wait complete. 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 Processor -- start 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 -- start start 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc51c19d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:58.949 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 0x7fc51c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c109ca0 con 0x7fc5040035c0 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c109b20 con 0x7fc51c197c60 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc5223b8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51c109e20 con 0x7fc51c19db90 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60048/0 (socket says 172.21.15.105:60048) 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 -- 172.21.15.105:0/327747878 learned_addr learned my addr 172.21.15.105:0/327747878 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 msgr2=0x7fc51c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 msgr2=0x7fc51c197720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc520955700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 0x7fc51c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:58.950 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51b7fe700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc51bfff700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc51c19d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc520955700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 msgr2=0x7fc51c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc520955700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 msgr2=0x7fc51c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.948+0000 7fc520955700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 0x7fc51c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc520955700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 0x7fc51c198110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc51bfff700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 msgr2=0x7fc51c197720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc51bfff700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc51bfff700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 msgr2=0x7fc51c198110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc51bfff700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 0x7fc51c198110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc51bfff700 1 -- 172.21.15.105:0/327747878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc51c1a4400 con 0x7fc5040035c0 2024-03-21T08:48:58.951 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc51bfff700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc51c19d650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc51c101b90 tx=0x7fc510005960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100052d0 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc510013e40 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51c1a4630 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc510011ba0 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.949+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51c1a4b70 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.950+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7fc510023070 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.950+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc51c1033a0 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.950+0000 7fc5197fa700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc5080396b0 0x7fc50803bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.950+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc51004e660 con 0x7fc5040035c0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.951+0000 7fc51b7fe700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc5080396b0 0x7fc50803bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:48:58.952 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.951+0000 7fc51b7fe700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc5080396b0 0x7fc50803bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009180 tx=0x7fc50c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:48:58.953 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:58.956+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc510028020 con 0x7fc5040035c0 2024-03-21T08:48:59.239 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.241+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc51c1a4dd0 con 0x7fc5040035c0 2024-03-21T08:48:59.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.242+0000 7fc5197fa700 1 -- 172.21.15.105:0/327747878 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1283 (secure 0 0 0) 0x7fc51001a390 con 0x7fc5040035c0 2024-03-21T08:48:59.240 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-03-21T08:48:59.240 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":3,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","modified":"2024-03-21T08:48:53.117146Z","created":"2024-03-21T08:47:01.466296Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:3300","nonce":0},{"type":"v1","addr":"172.21.15.87:6789","nonce":0}]},"addr":"172.21.15.87:6789/0","public_addr":"172.21.15.87:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:3300","nonce":0},{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:3300","nonce":0},{"type":"v1","addr":"172.21.15.100:6789","nonce":0}]},"addr":"172.21.15.100:6789/0","public_addr":"172.21.15.100:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-21T08:48:59.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc5080396b0 msgr2=0x7fc50803bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc5080396b0 0x7fc50803bb70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc50c009180 tx=0x7fc50c008040 comp rx=0 tx=0).stop 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 msgr2=0x7fc51c19d650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc51c19d650 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fc51c101b90 tx=0x7fc510005960 comp rx=0 tx=0).stop 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 shutdown_connections 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc5080396b0 0x7fc50803bb70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc51c197c60 0x7fc51c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc5040035c0 0x7fc51c19d650 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 --2- 172.21.15.105:0/327747878 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc51c19db90 0x7fc51c197720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 >> 172.21.15.105:0/327747878 conn(0x7fc51c0fd730 msgr2=0x7fc51c1058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 shutdown_connections 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:48:59.244+0000 7fc5223b8700 1 -- 172.21.15.105:0/327747878 wait complete. 2024-03-21T08:48:59.243 INFO:teuthology.orchestra.run.smithi105.stderr:dumped monmap epoch 3 2024-03-21T08:48:59.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:59.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:59 smithi105 ceph-mon[35993]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:59.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:59.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:48:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:48:59.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:48:59.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:59 smithi087 ceph-mon[32385]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:48:59.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:48:59.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:48:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:49:00.345 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-21T08:49:00.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph config generate-minimal-conf 2024-03-21T08:49:00.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:49:00.591 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:49:01.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: mon.b calling monitor election 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: mon.b calling monitor election 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: mon.c calling monitor election 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: mon.a calling monitor election 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: fsmap 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: mgrmap e13: x(active, since 61s) 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T08:49:01.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: Cluster is now healthy 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: overall HEALTH_OK 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T08:49:01.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:01.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:01.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:01.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: Reconfiguring daemon mon.a on smithi087 2024-03-21T08:49:01.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:01.411 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:49:01.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: mon.b calling monitor election 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: mon.b calling monitor election 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: mon.c calling monitor election 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: mon.a calling monitor election 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: fsmap 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: mgrmap e13: x(active, since 61s) 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: Cluster is now healthy 2024-03-21T08:49:01.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: overall HEALTH_OK 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:01.487 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: Reconfiguring daemon mon.a on smithi087 2024-03-21T08:49:01.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:01.488 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: mon.b calling monitor election 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: mon.b calling monitor election 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: mon.c calling monitor election 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: mon.a calling monitor election 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: monmap e3: 3 mons at {a=[v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0],b=[v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0],c=[v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-21T08:49:01.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: fsmap 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: osdmap e4: 0 total, 0 up, 0 in 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: mgrmap e13: x(active, since 61s) 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,c) 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: Cluster is now healthy 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: overall HEALTH_OK 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: Reconfiguring mon.a (unknown last config time)... 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: Reconfiguring daemon mon.a on smithi087 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:01.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:49:03.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 -- 172.21.15.87:0/2459896333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183406ab90 msgr2=0x7f183406af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:03.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 --2- 172.21.15.87:0/2459896333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183406ab90 0x7f183406af70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f1824009990 tx=0x7f1824009ca0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 -- 172.21.15.87:0/2459896333 shutdown_connections 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 --2- 172.21.15.87:0/2459896333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1834107550 0x7f183410f100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 --2- 172.21.15.87:0/2459896333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f183406ab90 0x7f183406af70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 --2- 172.21.15.87:0/2459896333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 0x7f1834106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 -- 172.21.15.87:0/2459896333 >> 172.21.15.87:0/2459896333 conn(0x7f1834078160 msgr2=0x7f1834078570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 -- 172.21.15.87:0/2459896333 shutdown_connections 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.356+0000 7f183bb0b700 1 -- 172.21.15.87:0/2459896333 wait complete. 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 Processor -- start 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 -- start start 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183406ab90 0x7f18341010c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 0x7f1834101600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 0x7f1834101b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834112510 con 0x7f1834107550 2024-03-21T08:49:03.355 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834112390 con 0x7f183406b540 2024-03-21T08:49:03.356 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.357+0000 7f183bb0b700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834112690 con 0x7f183406ab90 2024-03-21T08:49:03.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 0x7f1834101b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:03.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f18390a6700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 0x7f1834101600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:03.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 0x7f1834101b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:48434/0 (socket says 172.21.15.87:48434) 2024-03-21T08:49:03.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 -- 172.21.15.87:0/3665309042 learned_addr learned my addr 172.21.15.87:0/3665309042 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f18398a7700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183406ab90 0x7f18341010c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 -- 172.21.15.87:0/3665309042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183406ab90 msgr2=0x7f18341010c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183406ab90 0x7f18341010c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 -- 172.21.15.87:0/3665309042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 msgr2=0x7f1834101600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 0x7f1834101600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 -- 172.21.15.87:0/3665309042 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18341a6dc0 con 0x7f1834107550 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f18390a6700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 0x7f1834101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183a0a8700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 0x7f1834101b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b970 tx=0x7f182c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c00d860 con 0x7f1834107550 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f182c00e430 con 0x7f1834107550 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.358+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18341a70b0 con 0x7f1834107550 2024-03-21T08:49:03.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.359+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f182c017660 con 0x7f1834107550 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.359+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18341a75f0 con 0x7f1834107550 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.359+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f182c0203f0 con 0x7f1834107550 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.360+0000 7f182affd700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1820039460 0x7f182003b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.360+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f182c04bfb0 con 0x7f1834107550 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.360+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1834068e50 con 0x7f1834107550 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.360+0000 7f18398a7700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1820039460 0x7f182003b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:03.359 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.360+0000 7f18398a7700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1820039460 0x7f182003b920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f182400b5c0 tx=0x7f1824005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:03.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.365+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f182c013030 con 0x7f1834107550 2024-03-21T08:49:03.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.572+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f183402da30 con 0x7f1834107550 2024-03-21T08:49:03.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.573+0000 7f182affd700 1 -- 172.21.15.87:0/3665309042 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+275 (secure 0 0 0) 0x7f182c016360 con 0x7f1834107550 2024-03-21T08:49:03.570 INFO:teuthology.orchestra.run.smithi087.stdout:# minimal ceph.conf for 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:49:03.570 INFO:teuthology.orchestra.run.smithi087.stdout:[global] 2024-03-21T08:49:03.570 INFO:teuthology.orchestra.run.smithi087.stdout: fsid = 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:49:03.570 INFO:teuthology.orchestra.run.smithi087.stdout: mon_host = [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] 2024-03-21T08:49:03.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1820039460 msgr2=0x7f182003b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:03.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1820039460 0x7f182003b920 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f182400b5c0 tx=0x7f1824005960 comp rx=0 tx=0).stop 2024-03-21T08:49:03.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 msgr2=0x7f1834101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:03.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 0x7f1834101b40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f182c00b970 tx=0x7f182c00bc80 comp rx=0 tx=0).stop 2024-03-21T08:49:03.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 shutdown_connections 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1820039460 0x7f182003b920 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1834107550 0x7f1834101b40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f183406ab90 0x7f18341010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 --2- 172.21.15.87:0/3665309042 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f183406b540 0x7f1834101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 >> 172.21.15.87:0/3665309042 conn(0x7f1834078160 msgr2=0x7f18340fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 shutdown_connections 2024-03-21T08:49:03.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:03.575+0000 7f183bb0b700 1 -- 172.21.15.87:0/3665309042 wait complete. 2024-03-21T08:49:03.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:03.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:03.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:03.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: Reconfiguring mon.b (monmap changed)... 2024-03-21T08:49:03.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:03.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:03.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:03.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: Reconfiguring daemon mon.b on smithi100 2024-03-21T08:49:03.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3665309042' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:04.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: Reconfiguring mon.b (monmap changed)... 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: Reconfiguring daemon mon.b on smithi100 2024-03-21T08:49:04.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3665309042' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: Reconfiguring mon.b (monmap changed)... 2024-03-21T08:49:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: Reconfiguring daemon mon.b on smithi100 2024-03-21T08:49:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3665309042' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:04.249 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-21T08:49:04.249 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:49:04.249 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T08:49:04.279 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:49:04.279 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:49:04.346 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:49:04.346 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T08:49:04.377 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:49:04.377 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:49:04.444 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:49:04.444 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-03-21T08:49:04.475 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:49:04.475 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-21T08:49:04.543 INFO:tasks.cephadm:Adding mgr.x on smithi087 2024-03-21T08:49:04.543 INFO:tasks.cephadm:Adding mgr.y on smithi100 2024-03-21T08:49:04.543 INFO:tasks.cephadm:Adding mgr.z on smithi105 2024-03-21T08:49:04.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch apply mgr '3;smithi087=x;smithi100=y;smithi105=z' 2024-03-21T08:49:04.818 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:49:05.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: Reconfiguring mon.c (monmap changed)... 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: Reconfiguring daemon mon.c on smithi105 2024-03-21T08:49:05.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:05 smithi087 ceph-mon[32385]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: Reconfiguring mon.c (monmap changed)... 2024-03-21T08:49:05.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:05.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:05.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:05.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: Reconfiguring daemon mon.c on smithi105 2024-03-21T08:49:05.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:05 smithi105 ceph-mon[35993]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: Reconfiguring mon.c (monmap changed)... 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: Reconfiguring daemon mon.c on smithi105 2024-03-21T08:49:05.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:05 smithi100 ceph-mon[36306]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 -- 172.21.15.105:0/1444916009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9114102150 msgr2=0x7f9114102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 --2- 172.21.15.105:0/1444916009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9114102150 0x7f9114102570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9110009910 tx=0x7f9110009c20 comp rx=0 tx=0).stop 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 -- 172.21.15.105:0/1444916009 shutdown_connections 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 --2- 172.21.15.105:0/1444916009 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9114102150 0x7f9114102570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 -- 172.21.15.105:0/1444916009 >> 172.21.15.105:0/1444916009 conn(0x7f91140fb8e0 msgr2=0x7f91140fdd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 -- 172.21.15.105:0/1444916009 shutdown_connections 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 -- 172.21.15.105:0/1444916009 wait complete. 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 Processor -- start 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.163+0000 7f911b52f700 1 -- start start 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 0x7f9114100900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911419a250 0x7f911419c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:07.163 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911419cbd0 0x7f911419f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9114106c00 con 0x7f911419a250 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9114106a80 con 0x7f911419cbd0 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9114106d80 con 0x7f9114102150 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 0x7f9114100900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f9118aca700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911419a250 0x7f911419c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 0x7f9114100900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60050/0 (socket says 172.21.15.105:60050) 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 -- 172.21.15.105:0/2557925116 learned_addr learned my addr 172.21.15.105:0/2557925116 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 -- 172.21.15.105:0/2557925116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911419cbd0 msgr2=0x7f911419f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f9119acc700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911419cbd0 0x7f911419f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911419cbd0 0x7f911419f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 -- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911419a250 msgr2=0x7f911419c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911419a250 0x7f911419c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.164 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 -- 172.21.15.105:0/2557925116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911419f940 con 0x7f9114102150 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f9118aca700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911419a250 0x7f911419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f9119acc700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911419cbd0 0x7f911419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f91192cb700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 0x7f9114100900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9110000c00 tx=0x7f9110005000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9110027070 con 0x7f9114102150 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911419fbd0 con 0x7f9114102150 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9110022470 con 0x7f9114102150 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.164+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911000f680 con 0x7f9114102150 2024-03-21T08:49:07.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.165+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91141a00b0 con 0x7f9114102150 2024-03-21T08:49:07.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.165+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f911002a070 con 0x7f9114102150 2024-03-21T08:49:07.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.165+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91140637c0 con 0x7f9114102150 2024-03-21T08:49:07.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.165+0000 7f910a7fc700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91000396b0 0x7f910003bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:07.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.166+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f911004d430 con 0x7f9114102150 2024-03-21T08:49:07.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.166+0000 7f9118aca700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91000396b0 0x7f910003bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:07.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.166+0000 7f9118aca700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91000396b0 0x7f910003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9114100e60 tx=0x7f9104009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:07.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.171+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9110013070 con 0x7f9114102150 2024-03-21T08:49:07.376 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.378+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi100=y;smithi105=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9114062660 con 0x7f91000396b0 2024-03-21T08:49:07.383 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.385+0000 7f910a7fc700 1 -- 172.21.15.105:0/2557925116 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9114062660 con 0x7f91000396b0 2024-03-21T08:49:07.383 INFO:teuthology.orchestra.run.smithi105.stdout:Scheduled mgr update... 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91000396b0 msgr2=0x7f910003bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91000396b0 0x7f910003bb70 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9114100e60 tx=0x7f9104009210 comp rx=0 tx=0).stop 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 msgr2=0x7f9114100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 0x7f9114100900 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9110000c00 tx=0x7f9110005000 comp rx=0 tx=0).stop 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 shutdown_connections 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91000396b0 0x7f910003bb70 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911419cbd0 0x7f911419f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.385 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9114102150 0x7f9114100900 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 --2- 172.21.15.105:0/2557925116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911419a250 0x7f911419c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 >> 172.21.15.105:0/2557925116 conn(0x7f91140fb8e0 msgr2=0x7f91140fc510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 shutdown_connections 2024-03-21T08:49:07.386 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:49:07.387+0000 7f911b52f700 1 -- 172.21.15.105:0/2557925116 wait complete. 2024-03-21T08:49:08.007 DEBUG:teuthology.orchestra.run.smithi100:mgr.y> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.y.service 2024-03-21T08:49:08.010 DEBUG:teuthology.orchestra.run.smithi105:mgr.z> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.z.service 2024-03-21T08:49:08.013 INFO:tasks.cephadm:Deploying OSDs... 2024-03-21T08:49:08.013 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:49:08.013 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T08:49:08.029 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T08:49:08.029 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_1 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-03-21 08:48:12.132280286 +0000 2024-03-21T08:49:08.084 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-03-21 08:43:06.276911079 +0000 2024-03-21T08:49:08.085 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-03-21 08:43:06.276911079 +0000 2024-03-21T08:49:08.085 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-03-21 08:43:06.276911079 +0000 2024-03-21T08:49:08.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-03-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-03-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000214728 s, 2.4 MB/s 2024-03-21T08:49:08.152 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T08:49:08.209 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_2 2024-03-21T08:49:08.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:08.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:08.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:08.263 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T08:49:08.263 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-03-21 08:48:12.159279678 +0000 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-03-21 08:43:06.615903478 +0000 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-03-21 08:43:06.615903478 +0000 2024-03-21T08:49:08.264 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-03-21 08:43:06.615903478 +0000 2024-03-21T08:49:08.264 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T08:49:08.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:08.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:08.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:08.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:08.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:08.328 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-03-21T08:49:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-03-21T08:49:08.329 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.00019733 s, 2.6 MB/s 2024-03-21T08:49:08.330 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T08:49:08.386 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_3 2024-03-21T08:49:08.441 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-03-21 08:48:12.183279138 +0000 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-03-21 08:43:06.978895338 +0000 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-03-21 08:43:06.978895338 +0000 2024-03-21T08:49:08.442 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-03-21 08:43:06.978895338 +0000 2024-03-21T08:49:08.442 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T08:49:08.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-03-21T08:49:08.507 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-03-21T08:49:08.507 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000179024 s, 2.9 MB/s 2024-03-21T08:49:08.508 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T08:49:08.564 DEBUG:teuthology.orchestra.run.smithi087:> stat /dev/vg_nvme/lv_4 2024-03-21T08:49:08.619 INFO:teuthology.orchestra.run.smithi087.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T08:49:08.619 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:08.619 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-03-21T08:49:08.620 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:08.620 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:08.620 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-03-21 08:48:12.208278575 +0000 2024-03-21T08:49:08.620 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-03-21 08:43:07.333887378 +0000 2024-03-21T08:49:08.620 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-03-21 08:43:07.333887378 +0000 2024-03-21T08:49:08.620 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-03-21 08:43:07.333887378 +0000 2024-03-21T08:49:08.620 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records in 2024-03-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:1+0 records out 2024-03-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi087.stderr:512 bytes copied, 0.000197308 s, 2.6 MB/s 2024-03-21T08:49:08.686 DEBUG:teuthology.orchestra.run.smithi087:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T08:49:08.742 DEBUG:teuthology.orchestra.run.smithi100:> set -ex 2024-03-21T08:49:08.742 DEBUG:teuthology.orchestra.run.smithi100:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T08:49:08.761 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T08:49:08.761 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_1 2024-03-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 662 Links: 1 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 08:48:52.582884528 +0000 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 08:43:21.748374714 +0000 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 08:43:21.748374714 +0000 2024-03-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 08:43:21.748374714 +0000 2024-03-21T08:49:08.820 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T08:49:08.889 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T08:49:08.889 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T08:49:08.889 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000272407 s, 1.9 MB/s 2024-03-21T08:49:08.890 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T08:49:08.949 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_2 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 08:48:52.583884502 +0000 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 08:43:22.103365416 +0000 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 08:43:22.103365416 +0000 2024-03-21T08:49:09.006 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 08:43:22.103365416 +0000 2024-03-21T08:49:09.007 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T08:49:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T08:49:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T08:49:09.074 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000247451 s, 2.1 MB/s 2024-03-21T08:49:09.076 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T08:49:09.133 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_3 2024-03-21T08:49:09.188 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 699 Links: 1 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 08:48:52.583884502 +0000 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 08:43:22.444356484 +0000 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 08:43:22.444356484 +0000 2024-03-21T08:49:09.189 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 08:43:22.444356484 +0000 2024-03-21T08:49:09.189 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T08:49:09.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:08 smithi105 ceph-mon[35993]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi100=y;smithi105=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:09.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:08 smithi105 ceph-mon[35993]: Saving service mgr spec with placement smithi087=x;smithi100=y;smithi105=z;count:3 2024-03-21T08:49:09.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:08 smithi105 ceph-mon[35993]: Deploying daemon mgr.z on smithi105 2024-03-21T08:49:09.253 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:09 smithi100 ceph-mon[36306]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi100=y;smithi105=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:09.254 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:09 smithi100 ceph-mon[36306]: Saving service mgr spec with placement smithi087=x;smithi100=y;smithi105=z;count:3 2024-03-21T08:49:09.254 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:09 smithi100 ceph-mon[36306]: Deploying daemon mgr.z on smithi105 2024-03-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.00018591 s, 2.8 MB/s 2024-03-21T08:49:09.257 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T08:49:09.313 DEBUG:teuthology.orchestra.run.smithi100:> stat /dev/vg_nvme/lv_4 2024-03-21T08:49:09.368 INFO:teuthology.orchestra.run.smithi100.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:Access: 2024-03-21 08:48:52.583884502 +0000 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:Modify: 2024-03-21 08:43:22.776347789 +0000 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout:Change: 2024-03-21 08:43:22.776347789 +0000 2024-03-21T08:49:09.369 INFO:teuthology.orchestra.run.smithi100.stdout: Birth: 2024-03-21 08:43:22.776347789 +0000 2024-03-21T08:49:09.369 DEBUG:teuthology.orchestra.run.smithi100:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T08:49:09.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:08 smithi087 ceph-mon[32385]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi087=x;smithi100=y;smithi105=z", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:09.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:08 smithi087 ceph-mon[32385]: Saving service mgr spec with placement smithi087=x;smithi100=y;smithi105=z;count:3 2024-03-21T08:49:09.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:08 smithi087 ceph-mon[32385]: Deploying daemon mgr.z on smithi105 2024-03-21T08:49:09.435 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records in 2024-03-21T08:49:09.435 INFO:teuthology.orchestra.run.smithi100.stderr:1+0 records out 2024-03-21T08:49:09.435 INFO:teuthology.orchestra.run.smithi100.stderr:512 bytes copied, 0.000279437 s, 1.8 MB/s 2024-03-21T08:49:09.436 DEBUG:teuthology.orchestra.run.smithi100:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T08:49:09.491 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-03-21T08:49:09.492 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-03-21T08:49:09.507 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-21T08:49:09.507 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-03-21T08:49:09.562 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-21T08:49:09.562 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-21 08:49:09.466401700 +0000 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-21 08:43:46.995642301 +0000 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-21 08:43:46.995642301 +0000 2024-03-21T08:49:09.563 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-21 08:43:46.995642301 +0000 2024-03-21T08:49:09.563 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-21T08:49:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-21T08:49:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-21T08:49:09.710 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000252808 s, 2.0 MB/s 2024-03-21T08:49:09.711 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-21T08:49:09.727 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-03-21T08:49:09.782 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-21 08:49:09.658397850 +0000 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-21 08:43:47.335635395 +0000 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-21 08:43:47.335635395 +0000 2024-03-21T08:49:09.783 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-21 08:43:47.335635395 +0000 2024-03-21T08:49:09.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-21T08:49:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-21T08:49:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-21T08:49:09.851 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000191838 s, 2.7 MB/s 2024-03-21T08:49:09.852 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-21T08:49:09.909 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-03-21T08:49:09.965 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 701 Links: 1 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-21 08:49:09.658397850 +0000 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-21 08:43:47.675628489 +0000 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-21 08:43:47.675628489 +0000 2024-03-21T08:49:09.966 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-21 08:43:47.675628489 +0000 2024-03-21T08:49:09.967 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-21T08:49:10.031 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-21T08:49:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-21T08:49:10.032 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000207241 s, 2.5 MB/s 2024-03-21T08:49:10.033 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-21T08:49:10.089 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-03-21T08:49:10.107 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:10 smithi105 ceph-mon[35993]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:10.120 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-21T08:49:10.120 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-21T08:49:10.120 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 5h/5d Inode: 719 Links: 1 2024-03-21T08:49:10.120 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:49:10.120 INFO:teuthology.orchestra.run.smithi105.stdout:Context: system_u:object_r:device_t:s0 2024-03-21T08:49:10.121 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-03-21 08:49:09.659397830 +0000 2024-03-21T08:49:10.121 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-03-21 08:43:48.030621278 +0000 2024-03-21T08:49:10.121 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-03-21 08:43:48.030621278 +0000 2024-03-21T08:49:10.121 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: 2024-03-21 08:43:48.030621278 +0000 2024-03-21T08:49:10.121 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-21T08:49:10.186 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-03-21T08:49:10.186 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-03-21T08:49:10.186 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000194122 s, 2.6 MB/s 2024-03-21T08:49:10.187 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-21T08:49:10.243 INFO:tasks.cephadm:Deploying osd.0 on smithi087 with /dev/vg_nvme/lv_4... 2024-03-21T08:49:10.243 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T08:49:10.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:10 smithi100 ceph-mon[36306]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:10.271 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:09 smithi087 ceph-mon[32385]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:10.485 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:10 smithi105 systemd[1]: Started Ceph mgr.z for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:49:10.486 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:49:11.235 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:10.788+0000 7f4ab7e81200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:49:11.235 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:10.879+0000 7f4ab7e81200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:49:11.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: Deploying daemon mgr.y on smithi100 2024-03-21T08:49:11.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:11 smithi087 ceph-mon[32385]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:11.730 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:11.506+0000 7f4ab7e81200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:49:11.730 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:11.595+0000 7f4ab7e81200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:11.732+0000 7f4ab7e81200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: Deploying daemon mgr.y on smithi100 2024-03-21T08:49:11.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-mon[35993]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: Deploying daemon mgr.y on smithi100 2024-03-21T08:49:11.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:11 smithi100 ceph-mon[36306]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:11.985 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:11.824+0000 7f4ab7e81200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:49:11.985 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:11.904+0000 7f4ab7e81200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:49:12.393 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T08:49:12.394 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T08:49:12.394 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-03-21T08:49:12.394 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-03-21T08:49:12.394 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114477 s, 916 MB/s 2024-03-21T08:49:12.394 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-03-21T08:49:12.439 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:12.160+0000 7f4ab7e81200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:49:12.439 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:12.267+0000 7f4ab7e81200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:49:12.439 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:12.349+0000 7f4ab7e81200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:49:12.439 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:12.441+0000 7f4ab7e81200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:49:12.518 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 systemd[1]: Starting Ceph mgr.y for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:49:12.735 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:12.612+0000 7f4ab7e81200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:49:12.882 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 podman[37693]: 2024-03-21 08:49:12.508374009 +0000 UTC m=+0.017377152 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:49:12.886 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 podman[37693]: 2024-03-21 08:49:12.624741198 +0000 UTC m=+0.133744315 container create 5457a075ab1dabcaf51be4a515ca8459d41afd0b22903456f13b3a8ac5be85a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-03-21T08:49:12.886 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 podman[37693]: 2024-03-21 08:49:12.810382215 +0000 UTC m=+0.319385344 container init 5457a075ab1dabcaf51be4a515ca8459d41afd0b22903456f13b3a8ac5be85a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T08:49:12.886 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 podman[37693]: 2024-03-21 08:49:12.812941651 +0000 UTC m=+0.321944775 container start 5457a075ab1dabcaf51be4a515ca8459d41afd0b22903456f13b3a8ac5be85a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-03-21T08:49:12.886 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 ceph-mgr[37712]: -- 172.21.15.100:0/3139500912 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5588423245a0 con 0x5588422f9000 2024-03-21T08:49:12.886 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 bash[37693]: 5457a075ab1dabcaf51be4a515ca8459d41afd0b22903456f13b3a8ac5be85a1 2024-03-21T08:49:13.037 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_4 2024-03-21T08:49:13.268 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:12 smithi100 systemd[1]: Started Ceph mgr.y for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:49:13.314 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:49:13.768 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:13.379+0000 7f2b66498200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-21T08:49:13.768 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:13.475+0000 7f2b66498200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-21T08:49:13.958 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:13.586+0000 7f4ab7e81200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:49:13.958 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:13.730+0000 7f4ab7e81200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:14.058 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.235 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:14.059+0000 7f4ab7e81200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:49:14.235 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:14.149+0000 7f4ab7e81200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:14.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.256 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.164+0000 7f2b66498200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-21T08:49:14.256 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.258+0000 7f2b66498200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-21T08:49:14.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:14.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:14.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:14.521 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.407+0000 7f2b66498200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-21T08:49:14.522 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.506+0000 7f2b66498200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-21T08:49:14.536 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:14.350+0000 7f4ab7e81200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:49:14.536 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:14.538+0000 7f4ab7e81200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:49:14.850 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.589+0000 7f2b66498200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-21T08:49:14.851 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.853+0000 7f2b66498200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 -- 172.21.15.87:0/126072284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 msgr2=0x7f984c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 --2- 172.21.15.87:0/126072284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c103820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f983c0099c0 tx=0x7f983c009cd0 comp rx=0 tx=0).stop 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 -- 172.21.15.87:0/126072284 shutdown_connections 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 --2- 172.21.15.87:0/126072284 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 0x7f984c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 --2- 172.21.15.87:0/126072284 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 0x7f984c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 --2- 172.21.15.87:0/126072284 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c103820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 -- 172.21.15.87:0/126072284 >> 172.21.15.87:0/126072284 conn(0x7f984c0fd7a0 msgr2=0x7f984c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 -- 172.21.15.87:0/126072284 shutdown_connections 2024-03-21T08:49:14.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.895+0000 7f98517a4700 1 -- 172.21.15.87:0/126072284 wait complete. 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 Processor -- start 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 -- start start 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 0x7f984c198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c198940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 0x7f984c19f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c10de10 con 0x7f984c1033c0 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c10dc90 con 0x7f984c1021c0 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f98517a4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f984c10df90 con 0x7f984c103d60 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c198940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c198940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45120/0 (socket says 172.21.15.87:45120) 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 -- 172.21.15.87:0/463063008 learned_addr learned my addr 172.21.15.87:0/463063008 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984affd700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 0x7f984c198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984b7fe700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 0x7f984c19f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:14.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 -- 172.21.15.87:0/463063008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 msgr2=0x7f984c19f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 0x7f984c19f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 -- 172.21.15.87:0/463063008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 msgr2=0x7f984c198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 0x7f984c198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984a7fc700 1 -- 172.21.15.87:0/463063008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f984c1a0060 con 0x7f984c1033c0 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984b7fe700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 0x7f984c19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.896+0000 7f984affd700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 0x7f984c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.897+0000 7f984a7fc700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c198940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f983c00f690 tx=0x7f983c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.897+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c027070 con 0x7f984c1033c0 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.897+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f984c1a02f0 con 0x7f984c1033c0 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.897+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f983c00fb30 con 0x7f984c1033c0 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.897+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f983c017650 con 0x7f984c1033c0 2024-03-21T08:49:14.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.897+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f984c1a0750 con 0x7f984c1033c0 2024-03-21T08:49:14.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.898+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 13) v1 ==== 46350+0+0 (secure 0 0 0) 0x7f983c02a070 con 0x7f984c1033c0 2024-03-21T08:49:14.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.898+0000 7f9833fff700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9834039460 0x7f983403b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:14.899 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.898+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f983c04e310 con 0x7f984c1033c0 2024-03-21T08:49:14.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.898+0000 7f984affd700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9834039460 0x7f983403b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:14.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.898+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f984c068e50 con 0x7f984c1033c0 2024-03-21T08:49:14.900 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.898+0000 7f984affd700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9834039460 0x7f983403b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f984c103220 tx=0x7f9840009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:14.901 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:14.903+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f983c031070 con 0x7f984c1033c0 2024-03-21T08:49:15.068 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:14.818+0000 7f4ab7e81200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:49:15.068 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:15.071+0000 7f4ab7e81200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:49:15.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:15.120+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f984c0625d0 con 0x7f9834039460 2024-03-21T08:49:15.130 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:14.959+0000 7f2b66498200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-21T08:49:15.131 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:15.041+0000 7f2b66498200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-21T08:49:15.131 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:15.133+0000 7f2b66498200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-21T08:49:15.448 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:15.300+0000 7f4ab7e81200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:49:15.449 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:15.305+0000 7f2b66498200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-21T08:49:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.531 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: Reconfiguring daemon mgr.x on smithi087 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:49:15.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: Reconfiguring daemon mgr.x on smithi087 2024-03-21T08:49:15.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:15.737 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:49:15.737 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:49:15.737 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: Reconfiguring mgr.x (unknown last config time)... 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: Reconfiguring daemon mgr.x on smithi087 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:15.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:49:15.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:49:15.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:16.485 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:16.080+0000 7f4ab7e81200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:49:16.518 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:16.262+0000 7f2b66498200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-21T08:49:16.518 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:16.391+0000 7f2b66498200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-21T08:49:16.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:16.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:16.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:16.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:16.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.941 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:16.641+0000 7f4ab7e81200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:49:16.941 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:16.734+0000 7f4ab7e81200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:49:16.941 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z[37801]: 2024-03-21T08:49:16.944+0000 7f4ab7e81200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:49:16.941 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:16.941 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:16.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:16.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:16.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.989 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:16.989 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.989 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.989 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:16.990 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:16.990 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:16.990 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:16.990 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:16.717+0000 7f2b66498200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-21T08:49:16.990 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:16.802+0000 7f2b66498200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-21T08:49:16.990 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:16.991+0000 7f2b66498200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:49:17.268 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:17.168+0000 7f2b66498200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-21T08:49:17.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:17.569+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 14) v1 ==== 92074+0+0 (secure 0 0 0) 0x7f983c0177f0 con 0x7f984c1033c0 2024-03-21T08:49:17.653 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:17 smithi087 ceph-mon[32385]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:17.653 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:17 smithi087 ceph-mon[32385]: Standby manager daemon z started 2024-03-21T08:49:17.653 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:17 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:49:17.653 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:17 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:49:17.653 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:17 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:49:17.653 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:17 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-mon[36306]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-mon[36306]: Standby manager daemon z started 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:17.441+0000 7f2b66498200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-21T08:49:17.694 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:17.696+0000 7f2b66498200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-21T08:49:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:17 smithi105 ceph-mon[35993]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:17 smithi105 ceph-mon[35993]: Standby manager daemon z started 2024-03-21T08:49:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:17 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-03-21T08:49:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:17 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:49:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:17 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-03-21T08:49:17.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:17 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.105:0/2898282298' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:49:18.018 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:17.924+0000 7f2b66498200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-21T08:49:18.833 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:18 smithi087 ceph-mon[32385]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-21T08:49:18.833 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:18 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:49:18.833 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:18 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2486198973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "399009d1-377c-4c98-80d9-f0b22744ad40"}]: dispatch 2024-03-21T08:49:18.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:18 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2486198973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "399009d1-377c-4c98-80d9-f0b22744ad40"}]': finished 2024-03-21T08:49:18.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:18 smithi087 ceph-mon[32385]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T08:49:18.834 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:18 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:18.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:18 smithi105 ceph-mon[35993]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-21T08:49:18.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:18 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:49:18.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:18 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2486198973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "399009d1-377c-4c98-80d9-f0b22744ad40"}]: dispatch 2024-03-21T08:49:18.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:18 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2486198973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "399009d1-377c-4c98-80d9-f0b22744ad40"}]': finished 2024-03-21T08:49:18.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:18 smithi105 ceph-mon[35993]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T08:49:18.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:18 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:19.018 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:18.704+0000 7f2b66498200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-21T08:49:19.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-mon[36306]: mgrmap e14: x(active, since 78s), standbys: z 2024-03-21T08:49:19.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-03-21T08:49:19.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2486198973' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "399009d1-377c-4c98-80d9-f0b22744ad40"}]: dispatch 2024-03-21T08:49:19.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2486198973' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "399009d1-377c-4c98-80d9-f0b22744ad40"}]': finished 2024-03-21T08:49:19.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-mon[36306]: osdmap e5: 1 total, 0 up, 1 in 2024-03-21T08:49:19.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:18 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:19.518 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:19.259+0000 7f2b66498200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-21T08:49:19.518 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:19.352+0000 7f2b66498200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-21T08:49:19.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:19 smithi087 ceph-mon[32385]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:19.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:19 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1475315994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:49:19.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:19 smithi105 ceph-mon[35993]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:19.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:19 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1475315994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:49:20.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:19 smithi100 ceph-mon[36306]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:20.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:19 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1475315994' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:49:20.019 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 08:49:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y[37708]: 2024-03-21T08:49:19.561+0000 7f2b66498200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-21T08:49:20.586 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:20.587+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f983c017b70 con 0x7f984c1033c0 2024-03-21T08:49:20.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:20 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:49:20.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:20 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:49:20.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:20 smithi087 ceph-mon[32385]: Standby manager daemon y started 2024-03-21T08:49:20.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:20 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:49:20.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:20 smithi087 ceph-mon[32385]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:49:20.837 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:20 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:49:20.837 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:20 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:49:20.837 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:20 smithi105 ceph-mon[35993]: Standby manager daemon y started 2024-03-21T08:49:20.837 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:20 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:49:20.837 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:20 smithi105 ceph-mon[35993]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:49:20.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:20 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-03-21T08:49:20.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:20 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-21T08:49:20.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:20 smithi100 ceph-mon[36306]: Standby manager daemon y started 2024-03-21T08:49:20.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:20 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-03-21T08:49:20.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:20 smithi100 ceph-mon[36306]: from='mgr.? 172.21.15.100:0/3885401000' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-21T08:49:21.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:21 smithi087 ceph-mon[32385]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-21T08:49:21.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:49:21.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:21 smithi087 ceph-mon[32385]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:21.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:21 smithi105 ceph-mon[35993]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-21T08:49:21.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:49:21.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:21 smithi105 ceph-mon[35993]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:22.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:21 smithi100 ceph-mon[36306]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-03-21T08:49:22.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-21T08:49:22.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:21 smithi100 ceph-mon[36306]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:24.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:23 smithi087 ceph-mon[32385]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:24.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:23 smithi105 ceph-mon[35993]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:24.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:23 smithi100 ceph-mon[36306]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:26.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:25 smithi087 ceph-mon[32385]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:26.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T08:49:26.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:25 smithi105 ceph-mon[35993]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T08:49:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:25 smithi100 ceph-mon[36306]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-21T08:49:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:27.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:26 smithi087 ceph-mon[32385]: Deploying daemon osd.0 on smithi087 2024-03-21T08:49:27.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:26 smithi105 ceph-mon[35993]: Deploying daemon osd.0 on smithi087 2024-03-21T08:49:27.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:26 smithi100 ceph-mon[36306]: Deploying daemon osd.0 on smithi087 2024-03-21T08:49:28.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:27 smithi087 ceph-mon[32385]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:28.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:27 smithi105 ceph-mon[35993]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:28.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:27 smithi100 ceph-mon[36306]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:30.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:29 smithi087 ceph-mon[32385]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:30.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:29 smithi105 ceph-mon[35993]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:30.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:29 smithi100 ceph-mon[36306]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:32.009 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:31 smithi087 ceph-mon[32385]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:32.009 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:32.009 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:32.009 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:31 smithi105 ceph-mon[35993]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:32.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:31 smithi100 ceph-mon[36306]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:32.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:32.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:32.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:33 smithi087 ceph-mon[32385]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:34.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:34.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:34.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:33 smithi105 ceph-mon[35993]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:34.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:34.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:34.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:33 smithi100 ceph-mon[36306]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:34.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:34.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:34.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:34.877 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 0 on host 'smithi087' 2024-03-21T08:49:34.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.878+0000 7f9833fff700 1 -- 172.21.15.87:0/463063008 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f984c0625d0 con 0x7f9834039460 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9834039460 msgr2=0x7f983403b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9834039460 0x7f983403b920 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f984c103220 tx=0x7f9840009260 comp rx=0 tx=0).stop 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 msgr2=0x7f984c198940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c198940 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f983c00f690 tx=0x7f983c009580 comp rx=0 tx=0).stop 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 shutdown_connections 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9834039460 0x7f983403b920 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f984c1021c0 0x7f984c198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f984c103d60 0x7f984c19f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 --2- 172.21.15.87:0/463063008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f984c1033c0 0x7f984c198940 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 >> 172.21.15.87:0/463063008 conn(0x7f984c0fd7a0 msgr2=0x7f984c1066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:34.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 shutdown_connections 2024-03-21T08:49:34.880 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:49:34.881+0000 7f98517a4700 1 -- 172.21.15.87:0/463063008 wait complete. 2024-03-21T08:49:35.543 DEBUG:teuthology.orchestra.run.smithi087:osd.0> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.0.service 2024-03-21T08:49:35.546 INFO:tasks.cephadm:Deploying osd.1 on smithi100 with /dev/vg_nvme/lv_4... 2024-03-21T08:49:35.546 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T08:49:35.787 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:49:36.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:35 smithi087 ceph-mon[32385]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:36.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:36.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:36.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:36.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:35 smithi087 ceph-mon[32385]: from='osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:49:36.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:35 smithi087 ceph-mon[32385]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:49:36.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:35 smithi100 ceph-mon[36306]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:36.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:36.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:36.145 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:36.146 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:35 smithi100 ceph-mon[36306]: from='osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:49:36.146 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:35 smithi100 ceph-mon[36306]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:49:36.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:35 smithi105 ceph-mon[35993]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:36.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:36.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:36.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:36.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:35 smithi105 ceph-mon[35993]: from='osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:49:36.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:35 smithi105 ceph-mon[35993]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-21T08:49:37.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-mon[32385]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T08:49:37.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-mon[32385]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T08:49:37.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:37.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-mon[32385]: from='osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:49:37.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-mon[32385]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:49:37.097 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0[46874]: 2024-03-21T08:49:36.872+0000 7febb2035700 -1 osd.0 0 waiting for initial osdmap 2024-03-21T08:49:37.097 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:49:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0[46874]: 2024-03-21T08:49:36.874+0000 7febade28700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:49:37.152 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:36 smithi100 ceph-mon[36306]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T08:49:37.153 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:36 smithi100 ceph-mon[36306]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T08:49:37.153 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:36 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:37.153 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:36 smithi100 ceph-mon[36306]: from='osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:49:37.153 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:36 smithi100 ceph-mon[36306]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:49:37.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:36 smithi105 ceph-mon[35993]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-21T08:49:37.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:36 smithi105 ceph-mon[35993]: osdmap e6: 1 total, 0 up, 1 in 2024-03-21T08:49:37.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:36 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:37.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:36 smithi105 ceph-mon[35993]: from='osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:49:37.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:36 smithi105 ceph-mon[35993]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:49:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T08:49:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T08:49:37.709 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T08:49:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T08:49:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113296 s, 926 MB/s 2024-03-21T08:49:37.709 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T08:49:38.136 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:37 smithi105 ceph-mon[35993]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:38.137 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:37 smithi105 ceph-mon[35993]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:49:38.137 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:37 smithi105 ceph-mon[35993]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T08:49:38.137 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.137 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:37 smithi087 ceph-mon[32385]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:37 smithi087 ceph-mon[32385]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:49:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:37 smithi087 ceph-mon[32385]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T08:49:38.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:37 smithi100 ceph-mon[36306]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:37 smithi100 ceph-mon[36306]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:49:38.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:37 smithi100 ceph-mon[36306]: osdmap e7: 1 total, 0 up, 1 in 2024-03-21T08:49:38.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.416 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_4 2024-03-21T08:49:38.653 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:49:38.957 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:49:38.957 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:49:38.957 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] boot 2024-03-21T08:49:38.957 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:38.958 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] boot 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:39.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: osd.0 [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] boot 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: osdmap e8: 1 total, 1 up, 1 in 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:49:39.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:39.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:39.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:39.968 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:39 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi087 2024-03-21T08:49:39.968 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:39 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi087 to 17088M 2024-03-21T08:49:39.968 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:39 smithi100 ceph-mon[36306]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:39.968 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:39 smithi100 ceph-mon[36306]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T08:49:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:39 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi087 2024-03-21T08:49:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:39 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi087 to 17088M 2024-03-21T08:49:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:39 smithi087 ceph-mon[32385]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:39 smithi087 ceph-mon[32385]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T08:49:40.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:39 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi087 2024-03-21T08:49:40.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:39 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi087 to 17088M 2024-03-21T08:49:40.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:39 smithi105 ceph-mon[35993]: pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-21T08:49:40.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:39 smithi105 ceph-mon[35993]: osdmap e9: 1 total, 1 up, 1 in 2024-03-21T08:49:40.271 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 -- 172.21.15.100:0/4107401908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 msgr2=0x7f0c50109540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/4107401908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c50109540 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f0c4800b0d0 tx=0x7f0c4800b3e0 comp rx=0 tx=0).stop 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 -- 172.21.15.100:0/4107401908 shutdown_connections 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/4107401908 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c50109540 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/4107401908 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c501010f0 0x7f0c50101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/4107401908 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c501022f0 0x7f0c50102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 -- 172.21.15.100:0/4107401908 >> 172.21.15.100:0/4107401908 conn(0x7f0c500fc6b0 msgr2=0x7f0c500fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 -- 172.21.15.100:0/4107401908 shutdown_connections 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.212+0000 7f0c55dc7700 1 -- 172.21.15.100:0/4107401908 wait complete. 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 Processor -- start 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 -- start start 2024-03-21T08:49:40.272 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c501010f0 0x7f0c50198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c501022f0 0x7f0c50198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c5019f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5010cd30 con 0x7f0c501022f0 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5010cbb0 con 0x7f0c50102c90 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c55dc7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5010ceb0 con 0x7f0c501010f0 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c5019f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c5019f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:56720/0 (socket says 172.21.15.100:56720) 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 -- 172.21.15.100:0/3843130236 learned_addr learned my addr 172.21.15.100:0/3843130236 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4effd700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c501022f0 0x7f0c50198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4f7fe700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c501010f0 0x7f0c50198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 -- 172.21.15.100:0/3843130236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c501010f0 msgr2=0x7f0c50198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c501010f0 0x7f0c50198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:40.273 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 -- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c501022f0 msgr2=0x7f0c50198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c501022f0 0x7f0c50198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.213+0000 7f0c4ffff700 1 -- 172.21.15.100:0/3843130236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5019ff60 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.214+0000 7f0c4ffff700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c5019f8c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c480062a0 tx=0x7f0c48007d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.214+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c4801f070 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.214+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0c48004020 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.214+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c48004e60 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.214+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c501a01f0 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.214+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c501a0800 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.215+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c50068e50 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.216+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f0c480041d0 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.216+0000 7f0c4cff9700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c380a2fa0 0x7f0c380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.217+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f0c4800f020 con 0x7f0c50102c90 2024-03-21T08:49:40.274 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.217+0000 7f0c4f7fe700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c380a2fa0 0x7f0c380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:49:40.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.217+0000 7f0c4f7fe700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c380a2fa0 0x7f0c380a5460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005fd0 tx=0x7f0c40005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:49:40.275 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.220+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0c48028020 con 0x7f0c50102c90 2024-03-21T08:49:40.423 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:40.426+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c500fe720 con 0x7f0c380a2fa0 2024-03-21T08:49:41.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:49:41.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:49:41.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:41.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:49:41.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:49:41.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:41.249 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:49:41.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:49:41.250 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:41 smithi087 ceph-mon[32385]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:41 smithi087 ceph-mon[32385]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:42.174 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:41 smithi100 ceph-mon[36306]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:42.174 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:41 smithi100 ceph-mon[36306]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:42.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:41 smithi105 ceph-mon[35993]: from='client.24140 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:49:42.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:41 smithi105 ceph-mon[35993]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/2839453414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]: dispatch 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]: dispatch 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]': finished 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:49:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:43 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/2498240309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:49:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/2839453414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]: dispatch 2024-03-21T08:49:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]: dispatch 2024-03-21T08:49:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]': finished 2024-03-21T08:49:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T08:49:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:49:44.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:43 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/2498240309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:49:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/2839453414' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]: dispatch 2024-03-21T08:49:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]: dispatch 2024-03-21T08:49:44.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6"}]': finished 2024-03-21T08:49:44.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: osdmap e10: 2 total, 1 up, 2 in 2024-03-21T08:49:44.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:49:44.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:43 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/2498240309' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:49:46.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:45 smithi087 ceph-mon[32385]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:46.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:45 smithi105 ceph-mon[35993]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:46.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:45 smithi100 ceph-mon[36306]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:48.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:47 smithi087 ceph-mon[32385]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:48.181 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:47 smithi100 ceph-mon[36306]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:48.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:47 smithi105 ceph-mon[35993]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:50.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:49 smithi087 ceph-mon[32385]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:50.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:49 smithi105 ceph-mon[35993]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:50.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:49 smithi100 ceph-mon[36306]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:50.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:50 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T08:49:50.986 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:50 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:51.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:50 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T08:49:51.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:50 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:51.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:50 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-21T08:49:51.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:50 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:51 smithi100 ceph-mon[36306]: Deploying daemon osd.1 on smithi100 2024-03-21T08:49:52.142 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:51 smithi100 ceph-mon[36306]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:52.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:51 smithi105 ceph-mon[35993]: Deploying daemon osd.1 on smithi100 2024-03-21T08:49:52.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:51 smithi105 ceph-mon[35993]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:52.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:51 smithi087 ceph-mon[32385]: Deploying daemon osd.1 on smithi100 2024-03-21T08:49:52.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:51 smithi087 ceph-mon[32385]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:54.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:53 smithi105 ceph-mon[35993]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:54.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:53 smithi100 ceph-mon[36306]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:54.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:53 smithi087 ceph-mon[32385]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:56.171 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:55 smithi100 ceph-mon[36306]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:56.171 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:56.171 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:56.171 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:56.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:55 smithi105 ceph-mon[35993]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:56.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:56.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:56.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:56.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:55 smithi087 ceph-mon[32385]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:56.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:56.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:56.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:57.993 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:57 smithi100 ceph-mon[36306]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:58.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:57 smithi105 ceph-mon[35993]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:58.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:57 smithi087 ceph-mon[32385]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:58.935 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.937+0000 7f0c4cff9700 1 -- 172.21.15.100:0/3843130236 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0c500fe720 con 0x7f0c380a2fa0 2024-03-21T08:49:58.936 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 1 on host 'smithi100' 2024-03-21T08:49:58.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c380a2fa0 msgr2=0x7f0c380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:58.936 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c380a2fa0 0x7f0c380a5460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0c40005fd0 tx=0x7f0c40005e50 comp rx=0 tx=0).stop 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 msgr2=0x7f0c5019f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c5019f8c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0c480062a0 tx=0x7f0c48007d90 comp rx=0 tx=0).stop 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 shutdown_connections 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c380a2fa0 0x7f0c380a5460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0c50102c90 0x7f0c5019f8c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0c501010f0 0x7f0c50198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 --2- 172.21.15.100:0/3843130236 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0c501022f0 0x7f0c50198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.939+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 >> 172.21.15.100:0/3843130236 conn(0x7f0c500fc6b0 msgr2=0x7f0c500fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.940+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 shutdown_connections 2024-03-21T08:49:58.937 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:49:58.940+0000 7f0c55dc7700 1 -- 172.21.15.100:0/3843130236 wait complete. 2024-03-21T08:49:59.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:59.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:59.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.594 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:59.595 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.595 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.595 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:59.595 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:49:59.595 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:49:59 smithi100 ceph-mon[36306]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:49:59.619 DEBUG:teuthology.orchestra.run.smithi100:osd.1> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.1.service 2024-03-21T08:49:59.622 INFO:tasks.cephadm:Deploying osd.2 on smithi105 with /dev/vg_nvme/lv_4... 2024-03-21T08:49:59.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-21T08:49:59.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:59.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:59.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:59.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:59.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:49:59.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:49:59 smithi105 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:49:59.869 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:49:59.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:49:59.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:49:59.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:49:59 smithi087 ceph-mon[32385]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-21T08:50:01.037 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:00 smithi105 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T08:50:01.037 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:00 smithi105 ceph-mon[35993]: from='osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:50:01.038 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:00 smithi105 ceph-mon[35993]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T08:50:01.038 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:01.038 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:00 smithi105 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:50:01.038 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:00 smithi105 ceph-mon[35993]: overall HEALTH_OK 2024-03-21T08:50:01.147 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-mon[36306]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T08:50:01.147 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-mon[36306]: from='osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:50:01.147 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-mon[36306]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T08:50:01.147 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:01.147 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-mon[36306]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:50:01.148 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-mon[36306]: overall HEALTH_OK 2024-03-21T08:50:01.148 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1[40910]: 2024-03-21T08:50:00.899+0000 7efef842a700 -1 osd.1 0 waiting for initial osdmap 2024-03-21T08:50:01.148 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1[40910]: 2024-03-21T08:50:00.903+0000 7efef421d700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:50:01.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:00 smithi087 ceph-mon[32385]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-21T08:50:01.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:00 smithi087 ceph-mon[32385]: from='osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:50:01.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:00 smithi087 ceph-mon[32385]: osdmap e11: 2 total, 1 up, 2 in 2024-03-21T08:50:01.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:01.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:00 smithi087 ceph-mon[32385]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:50:01.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:00 smithi087 ceph-mon[32385]: overall HEALTH_OK 2024-03-21T08:50:01.597 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-21T08:50:01.597 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-21T08:50:01.598 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-21T08:50:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-21T08:50:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113876 s, 921 MB/s 2024-03-21T08:50:01.598 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 2024-03-21T08:50:01.598 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-21T08:50:02.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:01 smithi087 ceph-mon[32385]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:50:02.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:01 smithi087 ceph-mon[32385]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:50:02.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:01 smithi087 ceph-mon[32385]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T08:50:02.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.195 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:01 smithi105 ceph-mon[35993]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:50:02.195 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:01 smithi105 ceph-mon[35993]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:50:02.196 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:01 smithi105 ceph-mon[35993]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T08:50:02.196 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.196 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.196 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-03-21T08:50:02.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:01 smithi100 ceph-mon[36306]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:50:02.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:01 smithi100 ceph-mon[36306]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:50:02.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:01 smithi100 ceph-mon[36306]: osdmap e12: 2 total, 1 up, 2 in 2024-03-21T08:50:02.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:02.463 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] boot 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:03.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:03.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:03.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:02 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:50:03.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:50:03.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] boot 2024-03-21T08:50:03.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T08:50:03.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:03.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:03.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:03.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:03.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:02 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: osd.1 [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] boot 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: osdmap e13: 2 total, 2 up, 2 in 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:03.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:03.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:03.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:03.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:02 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:04.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 -- 172.21.15.105:0/446923033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89201010b0 msgr2=0x7f89201014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:04.046 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 --2- 172.21.15.105:0/446923033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89201010b0 0x7f89201014d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8918009990 tx=0x7f8918009ca0 comp rx=0 tx=0).stop 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 -- 172.21.15.105:0/446923033 shutdown_connections 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 --2- 172.21.15.105:0/446923033 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8920102c50 0x7f8920109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 --2- 172.21.15.105:0/446923033 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f89201010b0 0x7f89201014d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 --2- 172.21.15.105:0/446923033 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89201022b0 0x7f8920102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 -- 172.21.15.105:0/446923033 >> 172.21.15.105:0/446923033 conn(0x7f89200fc650 msgr2=0x7f89200fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 -- 172.21.15.105:0/446923033 shutdown_connections 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 -- 172.21.15.105:0/446923033 wait complete. 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 Processor -- start 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.045+0000 7f89281d3700 1 -- start start 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89201010b0 0x7f892006b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89201022b0 0x7f892019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 0x7f892019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:04.047 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892010ccf0 con 0x7f89201022b0 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892010cb70 con 0x7f89201010b0 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f892010ce70 con 0x7f8920102c50 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 0x7f892019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 0x7f892019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:60346/0 (socket says 172.21.15.105:60346) 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 -- 172.21.15.105:0/3940924185 learned_addr learned my addr 172.21.15.105:0/3940924185 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f892576e700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89201022b0 0x7f892019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8925f6f700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89201010b0 0x7f892006b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 -- 172.21.15.105:0/3940924185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89201010b0 msgr2=0x7f892006b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89201010b0 0x7f892006b290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 -- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89201022b0 msgr2=0x7f892019d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89201022b0 0x7f892019d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:04.048 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 -- 172.21.15.105:0/3940924185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89201a0000 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8926770700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 0x7f892019f730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b970 tx=0x7f891c00bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c019070 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89201a02f0 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f891c00ce10 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f891c007370 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.046+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89201a08b0 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.047+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8920068e50 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.048+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7f891c01c070 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.049+0000 7f8916ffd700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f890c0a2f20 0x7f890c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.049+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7f891c0cd370 con 0x7f8920102c50 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.049+0000 7f8925f6f700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f890c0a2f20 0x7f890c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:04.049 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.050+0000 7f8925f6f700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f890c0a2f20 0x7f890c0a53e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8918006010 tx=0x7f8918005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:50:04.050 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.053+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f891c09b9f0 con 0x7f8920102c50 2024-03-21T08:50:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:03 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi100 2024-03-21T08:50:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:03 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi100 to 17080M 2024-03-21T08:50:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:03 smithi105 ceph-mon[35993]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:50:04.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:03 smithi105 ceph-mon[35993]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T08:50:04.251 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:04.255+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f89200625d0 con 0x7f890c0a2f20 2024-03-21T08:50:04.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:03 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi100 2024-03-21T08:50:04.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:03 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi100 to 17080M 2024-03-21T08:50:04.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:03 smithi100 ceph-mon[36306]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:50:04.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:03 smithi100 ceph-mon[36306]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T08:50:04.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:03 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi100 2024-03-21T08:50:04.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:03 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi100 to 17080M 2024-03-21T08:50:04.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:03 smithi087 ceph-mon[32385]: pgmap v65: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-21T08:50:04.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:03 smithi087 ceph-mon[32385]: osdmap e14: 2 total, 2 up, 2 in 2024-03-21T08:50:05.132 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:05.133 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:05.133 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:05.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:05.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:05.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:05.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:05 smithi105 ceph-mon[35993]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:05.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:05 smithi105 ceph-mon[35993]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:06.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:05 smithi100 ceph-mon[36306]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:06.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:05 smithi100 ceph-mon[36306]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:06.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:05 smithi087 ceph-mon[32385]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:06.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:05 smithi087 ceph-mon[32385]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:07.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:06 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/2314859918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]: dispatch 2024-03-21T08:50:07.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:06 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]: dispatch 2024-03-21T08:50:07.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:06 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]': finished 2024-03-21T08:50:07.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:06 smithi105 ceph-mon[35993]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T08:50:07.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:06 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:07.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:06 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/2314859918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]: dispatch 2024-03-21T08:50:07.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:06 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]: dispatch 2024-03-21T08:50:07.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:06 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]': finished 2024-03-21T08:50:07.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:06 smithi100 ceph-mon[36306]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T08:50:07.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:06 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:07.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:06 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/2314859918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]: dispatch 2024-03-21T08:50:07.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:06 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]: dispatch 2024-03-21T08:50:07.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:06 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "90e5dc9a-effc-4556-a9a4-b84e24ae3338"}]': finished 2024-03-21T08:50:07.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:06 smithi087 ceph-mon[32385]: osdmap e15: 3 total, 2 up, 3 in 2024-03-21T08:50:07.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:06 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:08.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:07 smithi105 ceph-mon[35993]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:08.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:07 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/3802354953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:08.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:07 smithi100 ceph-mon[36306]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:08.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:07 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/3802354953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:08.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:07 smithi087 ceph-mon[32385]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:08.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:07 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/3802354953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:10.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:09 smithi105 ceph-mon[35993]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:10.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:09 smithi100 ceph-mon[36306]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:10.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:09 smithi087 ceph-mon[32385]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:12.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:11 smithi105 ceph-mon[35993]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:12.259 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:11 smithi100 ceph-mon[36306]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:12.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:11 smithi087 ceph-mon[32385]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:14.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:13 smithi105 ceph-mon[35993]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:14.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T08:50:14.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:13 smithi100 ceph-mon[36306]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T08:50:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:14.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:13 smithi087 ceph-mon[32385]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:14.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-21T08:50:14.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:15.128 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:14 smithi105 ceph-mon[35993]: Deploying daemon osd.2 on smithi105 2024-03-21T08:50:15.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:14 smithi100 ceph-mon[36306]: Deploying daemon osd.2 on smithi105 2024-03-21T08:50:15.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:14 smithi087 ceph-mon[32385]: Deploying daemon osd.2 on smithi105 2024-03-21T08:50:15.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:15 smithi105 ceph-mon[35993]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:16.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:15 smithi100 ceph-mon[36306]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:16.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:15 smithi087 ceph-mon[32385]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:17 smithi105 ceph-mon[35993]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:18.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:17 smithi100 ceph-mon[36306]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:18.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:17 smithi087 ceph-mon[32385]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:20.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:19 smithi087 ceph-mon[32385]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:20.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:19 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:20.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:19 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:20.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:19 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:20.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:19 smithi105 ceph-mon[35993]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:20.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:19 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:20.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:19 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:20.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:19 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:20.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:19 smithi100 ceph-mon[36306]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:20.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:19 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:20.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:19 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:20.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:19 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:22.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:21 smithi100 ceph-mon[36306]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:21 smithi105 ceph-mon[35993]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:21 smithi087 ceph-mon[32385]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:22.969 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 2 on host 'smithi105' 2024-03-21T08:50:22.970 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.972+0000 7f8916ffd700 1 -- 172.21.15.105:0/3940924185 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f89200625d0 con 0x7f890c0a2f20 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f890c0a2f20 msgr2=0x7f890c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f890c0a2f20 0x7f890c0a53e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8918006010 tx=0x7f8918005e80 comp rx=0 tx=0).stop 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 msgr2=0x7f892019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 0x7f892019f730 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f891c00b970 tx=0x7f891c00bc80 comp rx=0 tx=0).stop 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 shutdown_connections 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f890c0a2f20 0x7f890c0a53e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:22.972 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f89201010b0 0x7f892006b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:22.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8920102c50 0x7f892019f730 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:22.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 --2- 172.21.15.105:0/3940924185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f89201022b0 0x7f892019d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:22.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.974+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 >> 172.21.15.105:0/3940924185 conn(0x7f89200fc650 msgr2=0x7f89201055b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:50:22.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.975+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 shutdown_connections 2024-03-21T08:50:22.973 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:50:22.975+0000 7f89281d3700 1 -- 172.21.15.105:0/3940924185 wait complete. 2024-03-21T08:50:23.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:23.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:23.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:23.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:23.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:50:23.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:23 smithi105 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:23.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:50:23.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:23 smithi100 ceph-mon[36306]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:50:23.609 DEBUG:teuthology.orchestra.run.smithi105:osd.2> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.2.service 2024-03-21T08:50:23.611 INFO:tasks.cephadm:Deploying osd.3 on smithi087 with /dev/vg_nvme/lv_3... 2024-03-21T08:50:23.611 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T08:50:23.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:23.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:50:23.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:23 smithi087 ceph-mon[32385]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-21T08:50:23.856 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:50:25.126 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:24 smithi105 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T08:50:25.126 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:24 smithi105 ceph-mon[35993]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T08:50:25.126 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:24 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:25.126 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:24 smithi105 ceph-mon[35993]: from='osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:50:25.127 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:24 smithi105 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:50:25.245 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:24 smithi087 ceph-mon[32385]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T08:50:25.246 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:24 smithi087 ceph-mon[32385]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T08:50:25.246 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:24 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:25.246 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:24 smithi087 ceph-mon[32385]: from='osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:50:25.246 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:24 smithi087 ceph-mon[32385]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:50:25.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:24 smithi100 ceph-mon[36306]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-21T08:50:25.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:24 smithi100 ceph-mon[36306]: osdmap e16: 3 total, 2 up, 3 in 2024-03-21T08:50:25.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:24 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:25.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:24 smithi100 ceph-mon[36306]: from='osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:50:25.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:24 smithi100 ceph-mon[36306]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:50:25.735 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2[41158]: 2024-03-21T08:50:25.383+0000 7fdd74ce6700 -1 osd.2 0 waiting for initial osdmap 2024-03-21T08:50:25.735 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2[41158]: 2024-03-21T08:50:25.387+0000 7fdd6cad1700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:50:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T08:50:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T08:50:25.914 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-03-21T08:50:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-03-21T08:50:25.914 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115231 s, 910 MB/s 2024-03-21T08:50:25.914 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-03-21T08:50:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-mon[35993]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-mon[35993]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:50:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-mon[35993]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T08:50:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:25 smithi105 ceph-mon[35993]: from='osd.2 ' entity='osd.2' 2024-03-21T08:50:26.253 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:25 smithi087 ceph-mon[32385]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:26.254 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:25 smithi087 ceph-mon[32385]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:50:26.254 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:25 smithi087 ceph-mon[32385]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T08:50:26.254 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:26.254 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:26.254 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:25 smithi087 ceph-mon[32385]: from='osd.2 ' entity='osd.2' 2024-03-21T08:50:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:25 smithi100 ceph-mon[36306]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:25 smithi100 ceph-mon[36306]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:50:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:25 smithi100 ceph-mon[36306]: osdmap e17: 3 total, 2 up, 3 in 2024-03-21T08:50:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:25 smithi100 ceph-mon[36306]: from='osd.2 ' entity='osd.2' 2024-03-21T08:50:26.597 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_3 2024-03-21T08:50:26.876 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:50:27.323 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:50:27.323 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:50:27.323 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] boot 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.324 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:27.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:27.325 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] boot 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:27.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:27.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:27.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:27.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:50:27.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:50:27.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:27.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: osd.2 [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] boot 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: osdmap e18: 3 total, 3 up, 3 in 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:27.519 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:28.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi105 2024-03-21T08:50:28.303 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-21T08:50:28.304 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:28.304 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:50:28.304 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T08:50:28.304 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T08:50:28.304 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:28 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi105 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T08:50:28.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:28 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:50:28.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 -- 172.21.15.87:0/3098582403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 msgr2=0x7fcc8c103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:28.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 --2- 172.21.15.87:0/3098582403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c103820 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c0099c0 tx=0x7fcc7c009cd0 comp rx=0 tx=0).stop 2024-03-21T08:50:28.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 -- 172.21.15.87:0/3098582403 shutdown_connections 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 --2- 172.21.15.87:0/3098582403 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c103d60 0x7fcc8c10a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 --2- 172.21.15.87:0/3098582403 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc8c1021c0 0x7fcc8c1025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 --2- 172.21.15.87:0/3098582403 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c103820 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 -- 172.21.15.87:0/3098582403 >> 172.21.15.87:0/3098582403 conn(0x7fcc8c0fd7a0 msgr2=0x7fcc8c0ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 -- 172.21.15.87:0/3098582403 shutdown_connections 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 -- 172.21.15.87:0/3098582403 wait complete. 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.519+0000 7fcc944db700 1 Processor -- start 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 -- start start 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc8c1021c0 0x7fcc8c198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c1989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c103d60 0x7fcc8c19fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:28.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c10de20 con 0x7fcc8c1033c0 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c10dca0 con 0x7fcc8c103d60 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8c10dfa0 con 0x7fcc8c1021c0 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c1989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c1989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38526/0 (socket says 172.21.15.87:38526) 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 -- 172.21.15.87:0/1454787116 learned_addr learned my addr 172.21.15.87:0/1454787116 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc92a78700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c103d60 0x7fcc8c19fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc92277700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc8c1021c0 0x7fcc8c198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 -- 172.21.15.87:0/1454787116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc8c1021c0 msgr2=0x7fcc8c198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc8c1021c0 0x7fcc8c198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 -- 172.21.15.87:0/1454787116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c103d60 msgr2=0x7fcc8c19fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c103d60 0x7fcc8c19fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 -- 172.21.15.87:0/1454787116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8c1a00d0 con 0x7fcc8c1033c0 2024-03-21T08:50:28.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc91a76700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c1989b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b580 tx=0x7fcc7c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:50:28.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c027070 con 0x7fcc8c1033c0 2024-03-21T08:50:28.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcc7c0041d0 con 0x7fcc8c1033c0 2024-03-21T08:50:28.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc8c1a0360 con 0x7fcc8c1033c0 2024-03-21T08:50:28.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.520+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc7c00f460 con 0x7fcc8c1033c0 2024-03-21T08:50:28.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.521+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc8c1a0890 con 0x7fcc8c1033c0 2024-03-21T08:50:28.520 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi105 2024-03-21T08:50:28.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi105 to 17080M 2024-03-21T08:50:28.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-21T08:50:28.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:50:28.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-21T08:50:28.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: osdmap e19: 3 total, 3 up, 3 in 2024-03-21T08:50:28.521 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:28 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T08:50:28.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.521+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc8c068e50 con 0x7fcc8c1033c0 2024-03-21T08:50:28.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.523+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 15) v1 ==== 137798+0+0 (secure 0 0 0) 0x7fcc7c02a070 con 0x7fcc8c1033c0 2024-03-21T08:50:28.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.524+0000 7fcc837fe700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcc780a2fa0 0x7fcc780a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:28.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.524+0000 7fcc92277700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcc780a2fa0 0x7fcc780a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:28.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.524+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 3226+0+0 (secure 0 0 0) 0x7fcc7c0cf770 con 0x7fcc8c1033c0 2024-03-21T08:50:28.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.524+0000 7fcc92277700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcc780a2fa0 0x7fcc780a5460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc88006fd0 tx=0x7fcc88008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:50:28.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.528+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcc7c013070 con 0x7fcc8c1033c0 2024-03-21T08:50:28.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:28.731+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc8c0625d0 con 0x7fcc780a2fa0 2024-03-21T08:50:29.632 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51383]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:50:29.632 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51383]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:29.632 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51383]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:29.632 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51383]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:29.908 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51407]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:50:29.908 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51407]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:29.908 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51407]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:29.908 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 08:50:29 smithi087 sudo[51407]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T08:50:29.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T08:50:29.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:29.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:29.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:29.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:29.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:29 smithi087 ceph-mon[32385]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:29.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T08:50:29.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T08:50:29.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:29.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:29.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:29.963 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:29.964 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:29 smithi100 ceph-mon[36306]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: osdmap e20: 3 total, 3 up, 3 in 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:29.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:29 smithi105 ceph-mon[35993]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:30.216 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:29 smithi100 sudo[44753]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:50:30.216 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:29 smithi100 sudo[44753]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:30.216 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:29 smithi100 sudo[44753]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:30.216 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:29 smithi100 sudo[44753]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:30.518 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:30 smithi100 sudo[44756]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:50:30.518 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:30 smithi100 sudo[44756]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:30.518 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:30 smithi100 sudo[44756]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:30.518 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 08:50:30 smithi100 sudo[44756]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:30.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:30.648+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcc7c099af0 con 0x7fcc8c1033c0 2024-03-21T08:50:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:30 smithi105 ceph-mon[35993]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T08:50:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:30.735 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44801]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:50:30.735 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44801]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:30.735 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44801]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:30.735 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44801]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:30.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:30 smithi087 ceph-mon[32385]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T08:50:30.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:30.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:31.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:30 smithi100 ceph-mon[36306]: osdmap e21: 3 total, 3 up, 3 in 2024-03-21T08:50:31.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:31.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:31.235 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44804]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:50:31.235 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44804]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:31.235 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44804]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:31.235 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 08:50:30 smithi105 sudo[44804]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:31.279 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51451]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:50:31.279 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51451]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:31.279 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51451]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:31.279 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51451]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:31.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51478]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:50:31.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51478]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:31.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51478]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:31.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 sudo[51478]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:31.770 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44807]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:50:31.770 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44807]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:31.770 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44807]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:31.770 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44807]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:31.770 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44810]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:50:31.770 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44810]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:32.144 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T08:50:32.144 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:32.145 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:31 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3919196577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd94ec48-89ca-41e0-a0af-f91f7d8d3d14"}]: dispatch 2024-03-21T08:50:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44810]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 sudo[44810]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T08:50:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:32.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:32.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:31 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3919196577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd94ec48-89ca-41e0-a0af-f91f7d8d3d14"}]: dispatch 2024-03-21T08:50:32.271 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44759]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-03-21T08:50:32.272 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44759]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:32.272 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44759]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:32.272 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44759]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:32.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44762]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44762]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44762]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 sudo[44762]: pam_unix(sudo:session): session closed for user root 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: pgmap v87: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:32.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:32.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:32.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:32.770 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3919196577' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd94ec48-89ca-41e0-a0af-f91f7d8d3d14"}]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3919196577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd94ec48-89ca-41e0-a0af-f91f7d8d3d14"}]': finished 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:33.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:32 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/672082246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3919196577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd94ec48-89ca-41e0-a0af-f91f7d8d3d14"}]': finished 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:33.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:33.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:33.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:33.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:32 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/672082246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:33.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:33.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:33.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3919196577' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd94ec48-89ca-41e0-a0af-f91f7d8d3d14"}]': finished 2024-03-21T08:50:33.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: osdmap e22: 4 total, 3 up, 4 in 2024-03-21T08:50:33.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:33.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-21T08:50:33.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-21T08:50:33.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-21T08:50:33.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-21T08:50:33.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-21T08:50:33.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:32 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/672082246' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:34.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:33 smithi087 ceph-mon[32385]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:34.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:33 smithi105 ceph-mon[35993]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:34.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:33 smithi100 ceph-mon[36306]: pgmap v89: 1 pgs: 1 creating+peering; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:36.054 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:35 smithi087 ceph-mon[32385]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:36.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:35 smithi105 ceph-mon[35993]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:36.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:35 smithi100 ceph-mon[36306]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:37.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:37 smithi087 ceph-mon[32385]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:38.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:37 smithi105 ceph-mon[35993]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:37 smithi100 ceph-mon[36306]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:40.030 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:39 smithi087 ceph-mon[32385]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:40.030 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T08:50:40.030 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:40.218 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:39 smithi105 ceph-mon[35993]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:40.219 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:39 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T08:50:40.219 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:39 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:40.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:39 smithi100 ceph-mon[36306]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:40.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:39 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-21T08:50:40.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:39 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:40.918 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:40 smithi087 ceph-mon[32385]: Deploying daemon osd.3 on smithi087 2024-03-21T08:50:41.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:40 smithi105 ceph-mon[35993]: Deploying daemon osd.3 on smithi087 2024-03-21T08:50:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:40 smithi100 ceph-mon[36306]: Deploying daemon osd.3 on smithi087 2024-03-21T08:50:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:41 smithi087 ceph-mon[32385]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:42.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:41 smithi105 ceph-mon[35993]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:41 smithi100 ceph-mon[36306]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:44.125 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:43 smithi087 ceph-mon[32385]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:43 smithi105 ceph-mon[35993]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:43 smithi100 ceph-mon[36306]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:45.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:45 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:45.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:45 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:45.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:45 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:45.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:45 smithi105 ceph-mon[35993]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:46.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:45 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:46.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:45 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:46.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:45 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:46.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:45 smithi100 ceph-mon[36306]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:46.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:45 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:46.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:45 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:46.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:45 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:46.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:45 smithi087 ceph-mon[32385]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:48.130 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:47 smithi087 ceph-mon[32385]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:48.183 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.185+0000 7fcc837fe700 1 -- 172.21.15.87:0/1454787116 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcc8c0625d0 con 0x7fcc780a2fa0 2024-03-21T08:50:48.183 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 3 on host 'smithi087' 2024-03-21T08:50:48.184 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcc780a2fa0 msgr2=0x7fcc780a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcc780a2fa0 0x7fcc780a5460 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fcc88006fd0 tx=0x7fcc88008040 comp rx=0 tx=0).stop 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 msgr2=0x7fcc8c1989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c1989b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc7c00b580 tx=0x7fcc7c005e80 comp rx=0 tx=0).stop 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 shutdown_connections 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fcc780a2fa0 0x7fcc780a5460 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcc8c103d60 0x7fcc8c19fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcc8c1021c0 0x7fcc8c198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 --2- 172.21.15.87:0/1454787116 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcc8c1033c0 0x7fcc8c1989b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 >> 172.21.15.87:0/1454787116 conn(0x7fcc8c0fd7a0 msgr2=0x7fcc8c106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:50:48.185 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 shutdown_connections 2024-03-21T08:50:48.186 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:50:48.187+0000 7fcc944db700 1 -- 172.21.15.87:0/1454787116 wait complete. 2024-03-21T08:50:48.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:47 smithi105 ceph-mon[35993]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:48.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:47 smithi100 ceph-mon[36306]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:48.860 DEBUG:teuthology.orchestra.run.smithi087:osd.3> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.3.service 2024-03-21T08:50:48.863 INFO:tasks.cephadm:Deploying osd.4 on smithi100 with /dev/vg_nvme/lv_3... 2024-03-21T08:50:48.864 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T08:50:49.108 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:50:49.474 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.475 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.475 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:49 smithi100 ceph-mon[36306]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T08:50:49.475 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:49 smithi100 ceph-mon[36306]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:49.475 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.475 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:49 smithi105 ceph-mon[35993]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T08:50:49.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:49 smithi105 ceph-mon[35993]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:49.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.595 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.596 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.596 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:49 smithi087 ceph-mon[32385]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-21T08:50:49.596 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:49 smithi087 ceph-mon[32385]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:49.596 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:49.596 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:50.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:50 smithi105 ceph-mon[35993]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T08:50:50.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:50 smithi105 ceph-mon[35993]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T08:50:50.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:50 smithi105 ceph-mon[35993]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:50:50.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:50 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:50.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:50 smithi100 ceph-mon[36306]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T08:50:50.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:50 smithi100 ceph-mon[36306]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T08:50:50.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:50 smithi100 ceph-mon[36306]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:50:50.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:50 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:50.604 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:50 smithi087 ceph-mon[32385]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-21T08:50:50.605 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:50 smithi087 ceph-mon[32385]: osdmap e23: 4 total, 3 up, 4 in 2024-03-21T08:50:50.605 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:50 smithi087 ceph-mon[32385]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:50:50.605 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:50 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:50.605 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 08:50:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T08:50:50.189+0000 7f356fcf4700 -1 osd.3 0 waiting for initial osdmap 2024-03-21T08:50:50.605 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 08:50:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T08:50:50.195+0000 7f356bae7700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:50:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T08:50:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T08:50:51.060 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T08:50:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T08:50:51.060 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112281 s, 934 MB/s 2024-03-21T08:50:51.060 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T08:50:51.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:50:51.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T08:50:51.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.356 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:51.356 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] boot 2024-03-21T08:50:51.356 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T08:50:51.356 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] boot 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T08:50:51.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: from='osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: osdmap e24: 4 total, 3 up, 4 in 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: osd.3 [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] boot 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: osdmap e25: 4 total, 4 up, 4 in 2024-03-21T08:50:51.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-21T08:50:51.706 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_3 2024-03-21T08:50:51.942 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:50:52.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:52 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:50:52.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:52 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:50:52.355 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:52 smithi100 ceph-mon[36306]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T08:50:52.382 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:52 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:50:52.383 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:52 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:50:52.383 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:52 smithi087 ceph-mon[32385]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T08:50:52.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:52 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:50:52.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:52 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:50:52.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:52 smithi105 ceph-mon[35993]: osdmap e26: 4 total, 4 up, 4 in 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 -- 172.21.15.100:0/1743958942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 msgr2=0x7f0eec10a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/1743958942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec10a320 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0ee400b0a0 tx=0x7f0ee400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 -- 172.21.15.100:0/1743958942 shutdown_connections 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/1743958942 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec10a320 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/1743958942 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0eec102c50 0x7f0eec105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/1743958942 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eec105580 0x7f0eec107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 -- 172.21.15.100:0/1743958942 >> 172.21.15.100:0/1743958942 conn(0x7f0eec0fc660 msgr2=0x7f0eec0feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 -- 172.21.15.100:0/1743958942 shutdown_connections 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 -- 172.21.15.100:0/1743958942 wait complete. 2024-03-21T08:50:53.657 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 Processor -- start 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.658+0000 7f0ef19f7700 1 -- start start 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ef19f7700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0eec102c50 0x7f0eec198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ef19f7700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eec105580 0x7f0eec198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ef19f7700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec19f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ef19f7700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec10ddf0 con 0x7f0eec102c50 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ef19f7700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec10dc70 con 0x7f0eec107eb0 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ef19f7700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eec10df70 con 0x7f0eec105580 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec19f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec19f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:37124/0 (socket says 172.21.15.100:37124) 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 -- 172.21.15.100:0/2046897067 learned_addr learned my addr 172.21.15.100:0/2046897067 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0ee27fc700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eec105580 0x7f0eec198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeaffd700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0eec102c50 0x7f0eec198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:53.658 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 -- 172.21.15.100:0/2046897067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eec105580 msgr2=0x7f0eec198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eec105580 0x7f0eec198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 -- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0eec102c50 msgr2=0x7f0eec198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0eec102c50 0x7f0eec198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 -- 172.21.15.100:0/2046897067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eec1a0090 con 0x7f0eec107eb0 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.659+0000 7f0eeb7fe700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec19f9f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4007fa0 tx=0x7f0ee400be60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.660+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4021030 con 0x7f0eec107eb0 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.660+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0eec1a0320 con 0x7f0eec107eb0 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.660+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ee4007a80 con 0x7f0eec107eb0 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.660+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4004890 con 0x7f0eec107eb0 2024-03-21T08:50:53.659 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.660+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0eec1a07d0 con 0x7f0eec107eb0 2024-03-21T08:50:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.661+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0eec068e50 con 0x7f0eec107eb0 2024-03-21T08:50:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.662+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0ee4024070 con 0x7f0eec107eb0 2024-03-21T08:50:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.663+0000 7f0ef09f5700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0ed80a2fb0 0x7f0ed80a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:50:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.663+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 3698+0+0 (secure 0 0 0) 0x7f0ee400f020 con 0x7f0eec107eb0 2024-03-21T08:50:53.660 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.663+0000 7f0eeaffd700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0ed80a2fb0 0x7f0ed80a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:50:53.661 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.664+0000 7f0eeaffd700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0ed80a2fb0 0x7f0ed80a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005950 tx=0x7f0edc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:50:53.663 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.667+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ee409d260 con 0x7f0eec107eb0 2024-03-21T08:50:53.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi087 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi087 to 8544M 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:53.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:53.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:53.736 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:53 smithi105 ceph-mon[35993]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T08:50:53.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi087 2024-03-21T08:50:53.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:53.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:53.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi087 to 8544M 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:53.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:53 smithi100 ceph-mon[36306]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T08:50:53.871 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:50:53.874+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f0eec0625d0 con 0x7f0ed80a2fb0 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi087 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi087 to 8544M 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:53.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:53.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:53 smithi087 ceph-mon[32385]: osdmap e27: 4 total, 4 up, 4 in 2024-03-21T08:50:54.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:54 smithi100 ceph-mon[36306]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:54.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:54 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:54.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:54 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:54.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:54 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:54.769 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:54 smithi100 ceph-mon[36306]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T08:50:54.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:54 smithi087 ceph-mon[32385]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:54.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:54 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:54.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:54 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:54.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:54 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:54.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:54 smithi087 ceph-mon[32385]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T08:50:54.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:54 smithi105 ceph-mon[35993]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:50:54.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:54 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:50:54.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:54 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:50:54.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:54 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:50:54.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:54 smithi105 ceph-mon[35993]: osdmap e28: 4 total, 4 up, 4 in 2024-03-21T08:50:55.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:55 smithi100 ceph-mon[36306]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:55.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:55 smithi100 ceph-mon[36306]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T08:50:55.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:55 smithi087 ceph-mon[32385]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:55.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:55 smithi087 ceph-mon[32385]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T08:50:55.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:55 smithi105 ceph-mon[35993]: pgmap v106: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:55.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:55 smithi105 ceph-mon[35993]: osdmap e29: 4 total, 4 up, 4 in 2024-03-21T08:50:56.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:56 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/3099754307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]: dispatch 2024-03-21T08:50:56.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:56 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]: dispatch 2024-03-21T08:50:56.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:56 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]': finished 2024-03-21T08:50:56.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:56 smithi105 ceph-mon[35993]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T08:50:56.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:56 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:50:56.998 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:56 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/3099754307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]: dispatch 2024-03-21T08:50:56.999 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:56 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]: dispatch 2024-03-21T08:50:56.999 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:56 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]': finished 2024-03-21T08:50:56.999 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:56 smithi100 ceph-mon[36306]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T08:50:56.999 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:56 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:50:57.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:56 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/3099754307' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]: dispatch 2024-03-21T08:50:57.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:56 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]: dispatch 2024-03-21T08:50:57.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:56 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d"}]': finished 2024-03-21T08:50:57.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:56 smithi087 ceph-mon[32385]: osdmap e30: 5 total, 4 up, 5 in 2024-03-21T08:50:57.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:56 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:50:57.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:57 smithi105 ceph-mon[35993]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:57.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:57 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/1320315470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:58.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:57 smithi100 ceph-mon[36306]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:58.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:57 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/1320315470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:50:58.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:57 smithi087 ceph-mon[32385]: pgmap v109: 1 pgs: 1 remapped+peering; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:50:58.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:57 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/1320315470' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:00.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:50:59 smithi087 ceph-mon[32385]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-21T08:51:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:50:59 smithi105 ceph-mon[35993]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-21T08:51:00.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:50:59 smithi100 ceph-mon[36306]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-03-21T08:51:02.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:01 smithi087 ceph-mon[32385]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-21T08:51:02.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:01 smithi105 ceph-mon[35993]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-21T08:51:02.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:01 smithi100 ceph-mon[36306]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 87 KiB/s, 0 objects/s recovering 2024-03-21T08:51:03.908 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:03 smithi100 ceph-mon[36306]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:51:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:03 smithi087 ceph-mon[32385]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:51:04.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:03 smithi105 ceph-mon[35993]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-21T08:51:05.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T08:51:05.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:05.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:04 smithi087 ceph-mon[32385]: Deploying daemon osd.4 on smithi100 2024-03-21T08:51:05.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T08:51:05.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:05.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:04 smithi105 ceph-mon[35993]: Deploying daemon osd.4 on smithi100 2024-03-21T08:51:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-21T08:51:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:04 smithi100 ceph-mon[36306]: Deploying daemon osd.4 on smithi100 2024-03-21T08:51:06.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:05 smithi100 ceph-mon[36306]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-21T08:51:06.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:05 smithi087 ceph-mon[32385]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-21T08:51:06.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:05 smithi105 ceph-mon[35993]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 60 KiB/s, 0 objects/s recovering 2024-03-21T08:51:08.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:07 smithi100 ceph-mon[36306]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-21T08:51:08.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:07 smithi087 ceph-mon[32385]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-21T08:51:08.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:07 smithi105 ceph-mon[35993]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-21T08:51:10.113 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:09 smithi100 ceph-mon[36306]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-21T08:51:10.113 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:10.113 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:10.113 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:10.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:09 smithi087 ceph-mon[32385]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-21T08:51:10.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:10.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:10.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:10.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:09 smithi105 ceph-mon[35993]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-21T08:51:10.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:10.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:10.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:12.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:11 smithi087 ceph-mon[32385]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:12.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:11 smithi105 ceph-mon[35993]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:12.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:11 smithi100 ceph-mon[36306]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:12.837 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.840+0000 7f0ef09f5700 1 -- 172.21.15.100:0/2046897067 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f0eec0625d0 con 0x7f0ed80a2fb0 2024-03-21T08:51:12.837 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 4 on host 'smithi100' 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.842+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0ed80a2fb0 msgr2=0x7f0ed80a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.842+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0ed80a2fb0 0x7f0ed80a5470 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0edc005950 tx=0x7f0edc00a300 comp rx=0 tx=0).stop 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.842+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 msgr2=0x7f0eec19f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.842+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec19f9f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4007fa0 tx=0x7f0ee400be60 comp rx=0 tx=0).stop 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 shutdown_connections 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0ed80a2fb0 0x7f0ed80a5470 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0eec107eb0 0x7f0eec19f9f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0eec102c50 0x7f0eec198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 --2- 172.21.15.100:0/2046897067 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0eec105580 0x7f0eec198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:12.840 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 >> 172.21.15.100:0/2046897067 conn(0x7f0eec0fc660 msgr2=0x7f0eec103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:51:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 shutdown_connections 2024-03-21T08:51:12.841 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:51:12.843+0000 7f0ef19f7700 1 -- 172.21.15.100:0/2046897067 wait complete. 2024-03-21T08:51:13.485 DEBUG:teuthology.orchestra.run.smithi100:osd.4> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.4.service 2024-03-21T08:51:13.487 INFO:tasks.cephadm:Deploying osd.5 on smithi105 with /dev/vg_nvme/lv_3... 2024-03-21T08:51:13.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-21T08:51:13.730 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:51:14.082 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:14.082 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.082 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: from='osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:51:14.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:13 smithi105 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:51:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: from='osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:51:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:13 smithi087 ceph-mon[32385]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: from='osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:51:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:13 smithi100 ceph-mon[36306]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-21T08:51:14.997 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:14 smithi100 ceph-mon[36306]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T08:51:14.998 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:14 smithi100 ceph-mon[36306]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T08:51:14.998 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:14 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:14.998 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:14 smithi100 ceph-mon[36306]: from='osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:51:14.998 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:14 smithi100 ceph-mon[36306]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:51:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:14 smithi105 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T08:51:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:14 smithi105 ceph-mon[35993]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T08:51:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:14 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:14 smithi105 ceph-mon[35993]: from='osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:51:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:14 smithi105 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:51:15.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:14 smithi087 ceph-mon[32385]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-21T08:51:15.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:14 smithi087 ceph-mon[32385]: osdmap e31: 5 total, 4 up, 5 in 2024-03-21T08:51:15.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:14 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:15.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:14 smithi087 ceph-mon[32385]: from='osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:51:15.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:14 smithi087 ceph-mon[32385]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:51:15.768 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4[47564]: 2024-03-21T08:51:15.389+0000 7ff40c881700 -1 osd.4 0 waiting for initial osdmap 2024-03-21T08:51:15.768 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4[47564]: 2024-03-21T08:51:15.397+0000 7ff40466c700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:51:15.795 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-21T08:51:15.795 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-21T08:51:15.795 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-21T08:51:15.795 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-21T08:51:15.795 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112438 s, 933 MB/s 2024-03-21T08:51:15.796 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-21T08:51:16.109 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:15 smithi105 ceph-mon[35993]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:16.109 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:15 smithi105 ceph-mon[35993]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:51:16.109 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:15 smithi105 ceph-mon[35993]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T08:51:16.109 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.109 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.109 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:15 smithi105 ceph-mon[35993]: from='osd.4 ' entity='osd.4' 2024-03-21T08:51:16.182 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-mon[36306]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:16.183 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-mon[36306]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:51:16.183 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-mon[36306]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T08:51:16.183 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.183 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.183 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:15 smithi100 ceph-mon[36306]: from='osd.4 ' entity='osd.4' 2024-03-21T08:51:16.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:15 smithi087 ceph-mon[32385]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:16.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:15 smithi087 ceph-mon[32385]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:51:16.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:15 smithi087 ceph-mon[32385]: osdmap e32: 5 total, 4 up, 5 in 2024-03-21T08:51:16.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:15 smithi087 ceph-mon[32385]: from='osd.4 ' entity='osd.4' 2024-03-21T08:51:16.437 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-03-21T08:51:16.678 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:51:16.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:51:16.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] boot 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:16.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:16.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:16.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:16.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:16.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:16.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] boot 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:17.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: osd.4 [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] boot 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: osdmap e33: 5 total, 5 up, 5 in 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:17.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:17.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:17.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:17.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:17.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:17.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:18.076 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:17 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi100 2024-03-21T08:51:18.076 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:17 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi100 to 8540M 2024-03-21T08:51:18.076 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:17 smithi105 ceph-mon[35993]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:18.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:17 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi100 2024-03-21T08:51:18.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:17 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi100 to 8540M 2024-03-21T08:51:18.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:17 smithi100 ceph-mon[36306]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 -- 172.21.15.105:0/288362149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1010b0 msgr2=0x7f7aec1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:18.311 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 --2- 172.21.15.105:0/288362149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1010b0 0x7f7aec1014d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009990 tx=0x7f7adc009ca0 comp rx=0 tx=0).stop 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 -- 172.21.15.105:0/288362149 shutdown_connections 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 --2- 172.21.15.105:0/288362149 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec102c50 0x7f7aec109500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 --2- 172.21.15.105:0/288362149 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1010b0 0x7f7aec1014d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 --2- 172.21.15.105:0/288362149 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1022b0 0x7f7aec102710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 -- 172.21.15.105:0/288362149 >> 172.21.15.105:0/288362149 conn(0x7f7aec0fc670 msgr2=0x7f7aec0fea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 -- 172.21.15.105:0/288362149 shutdown_connections 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.299+0000 7f7af3351700 1 -- 172.21.15.105:0/288362149 wait complete. 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 Processor -- start 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 -- start start 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1010b0 0x7f7aec1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 0x7f7aec198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec102c50 0x7f7aec19f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:18.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec10cc90 con 0x7f7aec102c50 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec10cb10 con 0x7f7aec1010b0 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af3351700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aec10ce10 con 0x7f7aec1022b0 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 0x7f7aec198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 0x7f7aec198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:39802/0 (socket says 172.21.15.105:39802) 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 -- 172.21.15.105:0/3658344233 learned_addr learned my addr 172.21.15.105:0/3658344233 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af18ee700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec102c50 0x7f7aec19f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af10ed700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1010b0 0x7f7aec1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 -- 172.21.15.105:0/3658344233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1010b0 msgr2=0x7f7aec1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1010b0 0x7f7aec1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 -- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec102c50 msgr2=0x7f7aec19f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec102c50 0x7f7aec19f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:18.313 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af08ec700 1 -- 172.21.15.105:0/3658344233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec1a0020 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.300+0000 7f7af10ed700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1010b0 0x7f7aec1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.301+0000 7f7af08ec700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 0x7f7aec198900 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800d9b0 tx=0x7f7ae800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.301+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8017070 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.301+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7ae8005020 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.301+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aec1a0310 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.301+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae8009bd0 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.301+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aec1a0850 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.302+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aec068e50 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.303+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7ae80051d0 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.303+0000 7f7ae27fc700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7ad80a3030 0x7f7ad80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.304+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f7ae80ce5f0 con 0x7f7aec1022b0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.304+0000 7f7af10ed700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7ad80a3030 0x7f7ad80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:18.314 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.304+0000 7f7af10ed700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7ad80a3030 0x7f7ad80a54f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0039c0 tx=0x7f7adc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:51:18.315 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.307+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7ae8020020 con 0x7f7aec1022b0 2024-03-21T08:51:18.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:17 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi100 2024-03-21T08:51:18.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:17 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi100 to 8540M 2024-03-21T08:51:18.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:17 smithi087 ceph-mon[32385]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 111 MiB used, 358 GiB / 358 GiB avail 2024-03-21T08:51:18.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:18.512+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f7aec1a0b70 con 0x7f7ad80a3030 2024-03-21T08:51:19.220 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:18 smithi105 ceph-mon[35993]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T08:51:19.220 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:18 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:51:19.220 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:18 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:51:19.220 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:18 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:19.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:18 smithi100 ceph-mon[36306]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T08:51:19.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:18 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:51:19.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:18 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:51:19.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:18 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:19.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:18 smithi087 ceph-mon[32385]: osdmap e34: 5 total, 5 up, 5 in 2024-03-21T08:51:19.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:18 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:51:19.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:18 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:51:19.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:18 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:20.059 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:19 smithi105 ceph-mon[35993]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:51:20.059 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:19 smithi105 ceph-mon[35993]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:20.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:19 smithi100 ceph-mon[36306]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:51:20.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:19 smithi100 ceph-mon[36306]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:20.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:19 smithi087 ceph-mon[32385]: from='client.24214 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:51:20.309 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:19 smithi087 ceph-mon[32385]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/2490990721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]: dispatch 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]: dispatch 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]': finished 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:22.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:21 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/1338686218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/2490990721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]: dispatch 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]: dispatch 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]': finished 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:22.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:21 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/1338686218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:22.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/2490990721' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]: dispatch 2024-03-21T08:51:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]: dispatch 2024-03-21T08:51:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f908d67-699b-4d6f-8543-3f5dcaed7fcd"}]': finished 2024-03-21T08:51:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: osdmap e35: 6 total, 5 up, 6 in 2024-03-21T08:51:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:22.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:21 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/1338686218' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:24.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:23 smithi105 ceph-mon[35993]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:24.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:23 smithi100 ceph-mon[36306]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:24.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:23 smithi087 ceph-mon[32385]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:26.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:25 smithi105 ceph-mon[35993]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:26.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:25 smithi100 ceph-mon[36306]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:25 smithi087 ceph-mon[32385]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:27.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:27 smithi105 ceph-mon[35993]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:28.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:27 smithi100 ceph-mon[36306]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:27 smithi087 ceph-mon[32385]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:29.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:28 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T08:51:29.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:28 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:29.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:28 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T08:51:29.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:28 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:29.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:28 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-21T08:51:29.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:28 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:29 smithi105 ceph-mon[35993]: Deploying daemon osd.5 on smithi105 2024-03-21T08:51:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:29 smithi105 ceph-mon[35993]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:30.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:29 smithi100 ceph-mon[36306]: Deploying daemon osd.5 on smithi105 2024-03-21T08:51:30.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:29 smithi100 ceph-mon[36306]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:30.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:29 smithi087 ceph-mon[32385]: Deploying daemon osd.5 on smithi105 2024-03-21T08:51:30.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:29 smithi087 ceph-mon[32385]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:31.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:31 smithi105 ceph-mon[35993]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:32.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:31 smithi100 ceph-mon[36306]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:32.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:31 smithi087 ceph-mon[32385]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:34.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:33 smithi105 ceph-mon[35993]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:34.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:33 smithi100 ceph-mon[36306]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:34.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:33 smithi087 ceph-mon[32385]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:35.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:35.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:35.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:35.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:35 smithi105 ceph-mon[35993]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:35.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:35.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:35.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:35.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:35 smithi087 ceph-mon[32385]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:35.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:35.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:35.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:35.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:35 smithi100 ceph-mon[36306]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:37.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.825+0000 7f7ae27fc700 1 -- 172.21.15.105:0/3658344233 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7aec1a0b70 con 0x7f7ad80a3030 2024-03-21T08:51:37.823 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 5 on host 'smithi105' 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7ad80a3030 msgr2=0x7f7ad80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7ad80a3030 0x7f7ad80a54f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7adc0039c0 tx=0x7f7adc005fb0 comp rx=0 tx=0).stop 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 msgr2=0x7f7aec198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 0x7f7aec198900 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f7ae800d9b0 tx=0x7f7ae800dcc0 comp rx=0 tx=0).stop 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 shutdown_connections 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7ad80a3030 0x7f7ad80a54f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7aec1010b0 0x7f7aec1983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7aec102c50 0x7f7aec19f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 --2- 172.21.15.105:0/3658344233 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7aec1022b0 0x7f7aec198900 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:37.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 >> 172.21.15.105:0/3658344233 conn(0x7f7aec0fc670 msgr2=0x7f7aec0fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:51:37.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 shutdown_connections 2024-03-21T08:51:37.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:51:37.828+0000 7f7af3351700 1 -- 172.21.15.105:0/3658344233 wait complete. 2024-03-21T08:51:38.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:38.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:38.184 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:38.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:38.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:51:38.489 DEBUG:teuthology.orchestra.run.smithi105:osd.5> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.5.service 2024-03-21T08:51:38.491 INFO:tasks.cephadm:Deploying osd.6 on smithi087 with /dev/vg_nvme/lv_2... 2024-03-21T08:51:38.492 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T08:51:38.740 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:51:39.117 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:38 smithi087 ceph-mon[32385]: from='osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:51:39.117 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:38 smithi087 ceph-mon[32385]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:51:39.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:38 smithi105 ceph-mon[35993]: from='osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:51:39.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:38 smithi105 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:51:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:38 smithi100 ceph-mon[36306]: from='osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:51:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:38 smithi100 ceph-mon[36306]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-21T08:51:39.922 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:39 smithi105 ceph-mon[35993]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:39.923 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:39 smithi105 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T08:51:39.923 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:39 smithi105 ceph-mon[35993]: from='osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:51:39.923 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:39 smithi105 ceph-mon[35993]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T08:51:39.923 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:39 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:39.923 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:39 smithi105 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:51:40.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:39 smithi087 ceph-mon[32385]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:39 smithi087 ceph-mon[32385]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T08:51:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:39 smithi087 ceph-mon[32385]: from='osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:51:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:39 smithi087 ceph-mon[32385]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T08:51:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:40.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:39 smithi087 ceph-mon[32385]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:51:40.169 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:39 smithi100 ceph-mon[36306]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:40.169 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:39 smithi100 ceph-mon[36306]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-21T08:51:40.169 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:39 smithi100 ceph-mon[36306]: from='osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:51:40.169 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:39 smithi100 ceph-mon[36306]: osdmap e36: 6 total, 5 up, 6 in 2024-03-21T08:51:40.169 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:39 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:40.169 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:39 smithi100 ceph-mon[36306]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:51:40.485 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T08:51:40.392+0000 7fa69e2f0700 -1 osd.5 0 waiting for initial osdmap 2024-03-21T08:51:40.485 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T08:51:40.402+0000 7fa6960db700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:51:40.758 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T08:51:40.758 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T08:51:40.758 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-03-21T08:51:40.758 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-03-21T08:51:40.758 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115927 s, 905 MB/s 2024-03-21T08:51:40.758 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.087 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:40 smithi087 ceph-mon[32385]: from='osd.5 ' entity='osd.5' 2024-03-21T08:51:41.216 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:51:41.216 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T08:51:41.216 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.216 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.217 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T08:51:41.217 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.217 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:40 smithi105 ceph-mon[35993]: from='osd.5 ' entity='osd.5' 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: osdmap e37: 6 total, 5 up, 6 in 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: osdmap e38: 6 total, 5 up, 6 in 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:40 smithi100 ceph-mon[36306]: from='osd.5 ' entity='osd.5' 2024-03-21T08:51:41.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_2 2024-03-21T08:51:41.805 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:51:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:51:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] boot 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:41.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:41.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:51:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:51:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] boot 2024-03-21T08:51:42.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:42.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 447 GiB / 447 GiB avail 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: osd.5 [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] boot 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: osdmap e39: 6 total, 6 up, 6 in 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:42.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:51:42.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:42.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:42.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:51:42.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:51:43.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:42 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi105 2024-03-21T08:51:43.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:42 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-21T08:51:43.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:42 smithi087 ceph-mon[32385]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T08:51:43.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:42 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi105 2024-03-21T08:51:43.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:42 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-21T08:51:43.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:42 smithi105 ceph-mon[35993]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T08:51:43.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:42 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi105 2024-03-21T08:51:43.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:42 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi105 to 8540M 2024-03-21T08:51:43.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:42 smithi100 ceph-mon[36306]: osdmap e40: 6 total, 6 up, 6 in 2024-03-21T08:51:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 -- 172.21.15.87:0/2802201291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7164103d90 msgr2=0x7f716410a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2802201291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7164103d90 0x7f716410a640 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f715c00b0d0 tx=0x7f715c00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:51:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 -- 172.21.15.87:0/2802201291 shutdown_connections 2024-03-21T08:51:43.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2802201291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71641021f0 0x7f7164102610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2802201291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7164103d90 0x7f716410a640 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2802201291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f71641033f0 0x7f7164103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 -- 172.21.15.87:0/2802201291 >> 172.21.15.87:0/2802201291 conn(0x7f71640fd790 msgr2=0x7f71640ffbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:51:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 -- 172.21.15.87:0/2802201291 shutdown_connections 2024-03-21T08:51:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.472+0000 7f716a5ee700 1 -- 172.21.15.87:0/2802201291 wait complete. 2024-03-21T08:51:43.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 Processor -- start 2024-03-21T08:51:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 -- start start 2024-03-21T08:51:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 0x7f7164198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71641033f0 0x7f7164198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7164103d90 0x7f716419f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:43.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716410ddd0 con 0x7f71641021f0 2024-03-21T08:51:43.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716410dc50 con 0x7f71641033f0 2024-03-21T08:51:43.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f716a5ee700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716410df50 con 0x7f7164103d90 2024-03-21T08:51:43.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f7163fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 0x7f7164198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:43.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f7163fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 0x7f7164198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58516/0 (socket says 172.21.15.87:58516) 2024-03-21T08:51:43.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f7163fff700 1 -- 172.21.15.87:0/2736171624 learned_addr learned my addr 172.21.15.87:0/2736171624 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:51:43.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f71637fe700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71641033f0 0x7f7164198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f7163fff700 1 -- 172.21.15.87:0/2736171624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7164103d90 msgr2=0x7f716419f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.473+0000 7f7168b8b700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7164103d90 0x7f716419f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f7163fff700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7164103d90 0x7f716419f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f7163fff700 1 -- 172.21.15.87:0/2736171624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71641033f0 msgr2=0x7f7164198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f7163fff700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71641033f0 0x7f7164198860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f7163fff700 1 -- 172.21.15.87:0/2736171624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716419ff80 con 0x7f71641021f0 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f7168b8b700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7164103d90 0x7f716419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f7163fff700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 0x7f7164198320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f715400d9b0 tx=0x7f715400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:51:43.478 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154017030 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71641a0270 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7154004500 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154003aa0 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.474+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71641a07b0 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.475+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7164068e50 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.476+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f715401a070 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.476+0000 7f71617fa700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f714c0a2fb0 0x7f714c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.477+0000 7f71637fe700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f714c0a2fb0 0x7f714c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.477+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f71540ce280 con 0x7f71641021f0 2024-03-21T08:51:43.479 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.477+0000 7f71637fe700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f714c0a2fb0 0x7f714c0a5470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7158009b30 tx=0x7f7158009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:51:43.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.480+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f715409bf50 con 0x7f71641021f0 2024-03-21T08:51:43.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:51:43.687+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f71640625d0 con 0x7f714c0a2fb0 2024-03-21T08:51:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:43 smithi087 ceph-mon[32385]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:43 smithi087 ceph-mon[32385]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T08:51:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:51:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:51:44.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:43 smithi105 ceph-mon[35993]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:43 smithi105 ceph-mon[35993]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T08:51:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:51:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:51:44.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:43 smithi100 ceph-mon[36306]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:43 smithi100 ceph-mon[36306]: osdmap e41: 6 total, 6 up, 6 in 2024-03-21T08:51:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:51:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:51:44.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:44 smithi087 ceph-mon[32385]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:51:45.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:44 smithi087 ceph-mon[32385]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T08:51:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:44 smithi105 ceph-mon[35993]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:51:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:44 smithi105 ceph-mon[35993]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T08:51:45.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:44 smithi100 ceph-mon[36306]: from='client.14364 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:51:45.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:44 smithi100 ceph-mon[36306]: osdmap e42: 6 total, 6 up, 6 in 2024-03-21T08:51:46.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:45 smithi087 ceph-mon[32385]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:51:46.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:45 smithi087 ceph-mon[32385]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T08:51:46.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:45 smithi105 ceph-mon[35993]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:51:46.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:45 smithi105 ceph-mon[35993]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T08:51:46.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:45 smithi100 ceph-mon[36306]: pgmap v145: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:51:46.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:45 smithi100 ceph-mon[36306]: osdmap e43: 6 total, 6 up, 6 in 2024-03-21T08:51:47.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:46 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1911611258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa673f89-d652-4f14-9f54-c034e053a748"}]: dispatch 2024-03-21T08:51:47.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:46 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1911611258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa673f89-d652-4f14-9f54-c034e053a748"}]': finished 2024-03-21T08:51:47.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:46 smithi087 ceph-mon[32385]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T08:51:47.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:51:47.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:46 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1911611258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa673f89-d652-4f14-9f54-c034e053a748"}]: dispatch 2024-03-21T08:51:47.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:46 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1911611258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa673f89-d652-4f14-9f54-c034e053a748"}]': finished 2024-03-21T08:51:47.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:46 smithi105 ceph-mon[35993]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T08:51:47.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:51:47.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:46 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1911611258' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fa673f89-d652-4f14-9f54-c034e053a748"}]: dispatch 2024-03-21T08:51:47.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:46 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1911611258' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fa673f89-d652-4f14-9f54-c034e053a748"}]': finished 2024-03-21T08:51:47.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:46 smithi100 ceph-mon[36306]: osdmap e44: 7 total, 6 up, 7 in 2024-03-21T08:51:47.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:51:48.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:47 smithi105 ceph-mon[35993]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:51:48.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:47 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1106170492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:48.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:47 smithi100 ceph-mon[36306]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:51:48.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:47 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1106170492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:47 smithi087 ceph-mon[32385]: pgmap v148: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:51:48.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:47 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1106170492' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:51:50.191 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:49 smithi087 ceph-mon[32385]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T08:51:50.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:49 smithi105 ceph-mon[35993]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T08:51:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:49 smithi100 ceph-mon[36306]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T08:51:52.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:51 smithi105 ceph-mon[35993]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:51 smithi100 ceph-mon[36306]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:52.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:51 smithi087 ceph-mon[32385]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:54.027 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:53 smithi087 ceph-mon[32385]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:54.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:53 smithi105 ceph-mon[35993]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:54.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:53 smithi100 ceph-mon[36306]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:55.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:54 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T08:51:55.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:54 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:55.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:54 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T08:51:55.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:54 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:55.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:54 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-21T08:51:55.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:54 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:51:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:55 smithi105 ceph-mon[35993]: Deploying daemon osd.6 on smithi087 2024-03-21T08:51:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:55 smithi105 ceph-mon[35993]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:56.236 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:55 smithi087 ceph-mon[32385]: Deploying daemon osd.6 on smithi087 2024-03-21T08:51:56.236 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:55 smithi087 ceph-mon[32385]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:55 smithi100 ceph-mon[36306]: Deploying daemon osd.6 on smithi087 2024-03-21T08:51:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:55 smithi100 ceph-mon[36306]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:58.062 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:57 smithi087 ceph-mon[32385]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:57 smithi105 ceph-mon[35993]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:51:58.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:57 smithi100 ceph-mon[36306]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:00.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:59 smithi087 ceph-mon[32385]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:00.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:00.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:00.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:51:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:59 smithi105 ceph-mon[35993]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:51:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:59 smithi100 ceph-mon[36306]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:00.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:00.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:51:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:02.232 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:01 smithi087 ceph-mon[32385]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:02.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:01 smithi105 ceph-mon[35993]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:02.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:01 smithi100 ceph-mon[36306]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:03.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.567+0000 7f71617fa700 1 -- 172.21.15.87:0/2736171624 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f71640625d0 con 0x7f714c0a2fb0 2024-03-21T08:52:03.564 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 6 on host 'smithi087' 2024-03-21T08:52:03.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f714c0a2fb0 msgr2=0x7f714c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:03.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f714c0a2fb0 0x7f714c0a5470 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f7158009b30 tx=0x7f7158009260 comp rx=0 tx=0).stop 2024-03-21T08:52:03.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 msgr2=0x7f7164198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:03.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 0x7f7164198320 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f715400d9b0 tx=0x7f715400dd70 comp rx=0 tx=0).stop 2024-03-21T08:52:03.566 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 shutdown_connections 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f714c0a2fb0 0x7f714c0a5470 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7164103d90 0x7f716419f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f71641021f0 0x7f7164198320 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 --2- 172.21.15.87:0/2736171624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f71641033f0 0x7f7164198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 >> 172.21.15.87:0/2736171624 conn(0x7f71640fd790 msgr2=0x7f7164106620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 shutdown_connections 2024-03-21T08:52:03.567 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:52:03.570+0000 7f716a5ee700 1 -- 172.21.15.87:0/2736171624 wait complete. 2024-03-21T08:52:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:03 smithi087 ceph-mon[32385]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:03 smithi087 ceph-mon[32385]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T08:52:04.215 DEBUG:teuthology.orchestra.run.smithi087:osd.6> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.6.service 2024-03-21T08:52:04.218 INFO:tasks.cephadm:Deploying osd.7 on smithi100 with /dev/vg_nvme/lv_2... 2024-03-21T08:52:04.218 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T08:52:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:03 smithi105 ceph-mon[35993]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:04.236 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:03 smithi105 ceph-mon[35993]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T08:52:04.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:03 smithi100 ceph-mon[36306]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:04.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:04.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:04.248 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:03 smithi100 ceph-mon[36306]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-21T08:52:04.464 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:52:05.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:05.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:05.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T08:52:05.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T08:52:05.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:52:05.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:05.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:05 smithi100 ceph-mon[36306]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:05.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:05.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:05.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T08:52:05.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T08:52:05.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:52:05.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:05.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-mon[32385]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:05.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:05.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-21T08:52:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: osdmap e45: 7 total, 6 up, 7 in 2024-03-21T08:52:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:52:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:05.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:05 smithi105 ceph-mon[35993]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:06.172 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T08:52:05.976+0000 7f5e69ced700 -1 osd.6 0 waiting for initial osdmap 2024-03-21T08:52:06.172 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 08:52:05 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T08:52:05.985+0000 7f5e61ad8700 -1 osd.6 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:52:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T08:52:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T08:52:06.569 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T08:52:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T08:52:06.569 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129638 s, 809 MB/s 2024-03-21T08:52:06.569 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T08:52:06.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:06 smithi087 ceph-mon[32385]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:52:06.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:06 smithi087 ceph-mon[32385]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T08:52:06.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:06 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:06.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:06 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:06.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:06 smithi087 ceph-mon[32385]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' 2024-03-21T08:52:06.954 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:06 smithi100 ceph-mon[36306]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:52:06.954 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:06 smithi100 ceph-mon[36306]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T08:52:06.954 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:06 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:06.954 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:06 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:06.954 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:06 smithi100 ceph-mon[36306]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' 2024-03-21T08:52:06.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:06 smithi105 ceph-mon[35993]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:52:06.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:06 smithi105 ceph-mon[35993]: osdmap e46: 7 total, 6 up, 7 in 2024-03-21T08:52:06.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:06 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:06.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:06 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:06.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:06 smithi105 ceph-mon[35993]: from='osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078]' entity='osd.6' 2024-03-21T08:52:07.309 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_2 2024-03-21T08:52:07.601 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:52:07.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:52:07.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:52:07.884 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:07.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:07.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] boot 2024-03-21T08:52:07.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T08:52:07.885 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:07.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:52:07.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:52:07.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:07.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:07.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] boot 2024-03-21T08:52:07.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T08:52:07.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:07.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:52:07.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:52:07.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:07.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 173 MiB used, 536 GiB / 536 GiB avail 2024-03-21T08:52:07.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: osd.6 [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] boot 2024-03-21T08:52:07.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: osdmap e47: 7 total, 7 up, 7 in 2024-03-21T08:52:07.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-21T08:52:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 -- 172.21.15.100:0/1150114092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 msgr2=0x7f9dd01058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/1150114092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd01058b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4009990 tx=0x7f9dc4009ca0 comp rx=0 tx=0).stop 2024-03-21T08:52:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 -- 172.21.15.100:0/1150114092 shutdown_connections 2024-03-21T08:52:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/1150114092 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd0105df0 0x7f9dd0106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/1150114092 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd0100ab0 0x7f9dd0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:09.304 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/1150114092 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd01058b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 -- 172.21.15.100:0/1150114092 >> 172.21.15.100:0/1150114092 conn(0x7f9dd00fc6f0 msgr2=0x7f9dd00feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 -- 172.21.15.100:0/1150114092 shutdown_connections 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 -- 172.21.15.100:0/1150114092 wait complete. 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 Processor -- start 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 -- start start 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.305+0000 7f9dd7d36700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd0100ab0 0x7f9dd019e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd7d36700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd7d36700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd0105df0 0x7f9dd01a5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd7d36700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0114380 con 0x7f9dd0105df0 2024-03-21T08:52:09.305 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd7d36700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0114200 con 0x7f9dd0103420 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd7d36700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dd0114500 con 0x7f9dd0100ab0 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:57640/0 (socket says 172.21.15.100:57640) 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 -- 172.21.15.100:0/3153007607 learned_addr learned my addr 172.21.15.100:0/3153007607 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd5ad2700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd0100ab0 0x7f9dd019e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd62d3700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd0105df0 0x7f9dd01a5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 -- 172.21.15.100:0/3153007607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd0100ab0 msgr2=0x7f9dd019e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd0100ab0 0x7f9dd019e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:09.306 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 -- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd0105df0 msgr2=0x7f9dd01a5f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd0105df0 0x7f9dd01a5f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 -- 172.21.15.100:0/3153007607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dd01a65e0 con 0x7f9dd0103420 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd52d1700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd019ef10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005910 tx=0x7f9dc40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc4027070 con 0x7f9dd0103420 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9dc4004210 con 0x7f9dd0103420 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dc400f460 con 0x7f9dd0103420 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.306+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dd01a6870 con 0x7f9dd0103420 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.307+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dd01a6e50 con 0x7f9dd0103420 2024-03-21T08:52:09.307 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.308+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dd0068e50 con 0x7f9dd0103420 2024-03-21T08:52:09.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.308+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9dc402a070 con 0x7f9dd0103420 2024-03-21T08:52:09.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.309+0000 7f9dceffd700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9db80a3030 0x7f9db80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:09.308 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.309+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 4965+0+0 (secure 0 0 0) 0x7f9dc40cf440 con 0x7f9dd0103420 2024-03-21T08:52:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.309+0000 7f9dd5ad2700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9db80a3030 0x7f9db80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:09.309 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.310+0000 7f9dd5ad2700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9db80a3030 0x7f9db80a54f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8005950 tx=0x7f9dc800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:52:09.310 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.314+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9dc4013070 con 0x7f9dd0103420 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi087 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi087 to 5696M 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T08:52:09.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:09 smithi105 ceph-mon[35993]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:09.514 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:09.517+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f9dd00625d0 con 0x7f9db80a3030 2024-03-21T08:52:09.514 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi087 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi087 to 5696M 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T08:52:09.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:09 smithi100 ceph-mon[36306]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi087 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi087 to 5696M 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:09.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: osdmap e48: 7 total, 7 up, 7 in 2024-03-21T08:52:09.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:09 smithi087 ceph-mon[32385]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:10.428 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:10 smithi100 ceph-mon[36306]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:52:10.428 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:10 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:52:10.428 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:10 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:52:10.428 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:10 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:10.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:10 smithi105 ceph-mon[35993]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:52:10.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:10 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:52:10.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:10 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:52:10.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:10 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:10.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:10 smithi087 ceph-mon[32385]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:52:10.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:10 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:52:10.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:10 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:52:10.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:10 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:11.341 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:11 smithi100 ceph-mon[36306]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:11.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:11 smithi105 ceph-mon[35993]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:11.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:11 smithi087 ceph-mon[32385]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:12.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:12 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/2082377982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]: dispatch 2024-03-21T08:52:12.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:12 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]: dispatch 2024-03-21T08:52:12.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:12 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]': finished 2024-03-21T08:52:12.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:12 smithi087 ceph-mon[32385]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T08:52:12.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:12.704 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:12 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/2082377982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]: dispatch 2024-03-21T08:52:12.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:12 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]: dispatch 2024-03-21T08:52:12.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:12 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]': finished 2024-03-21T08:52:12.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:12 smithi100 ceph-mon[36306]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T08:52:12.705 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:12.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:12 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/2082377982' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]: dispatch 2024-03-21T08:52:12.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:12 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]: dispatch 2024-03-21T08:52:12.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:12 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "75a7a975-6491-4d37-b314-c75617e3b0cc"}]': finished 2024-03-21T08:52:12.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:12 smithi105 ceph-mon[35993]: osdmap e49: 8 total, 7 up, 8 in 2024-03-21T08:52:12.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:13.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:13 smithi087 ceph-mon[32385]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:13.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:13 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/2017605368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:52:13.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:13 smithi105 ceph-mon[35993]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:13.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:13 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/2017605368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:52:13.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:13 smithi100 ceph-mon[36306]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:13.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:13 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/2017605368' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:52:16.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:15 smithi087 ceph-mon[32385]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:16.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:15 smithi105 ceph-mon[35993]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:16.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:15 smithi100 ceph-mon[36306]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:18.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:17 smithi087 ceph-mon[32385]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:18.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:17 smithi105 ceph-mon[35993]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:18.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:17 smithi100 ceph-mon[36306]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:20.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:19 smithi087 ceph-mon[32385]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:20.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:19 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T08:52:20.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:19 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:20.191 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:19 smithi105 ceph-mon[35993]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:20.191 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:19 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T08:52:20.191 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:19 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:20.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:19 smithi100 ceph-mon[36306]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:20.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:19 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-21T08:52:20.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:19 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:21.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:20 smithi100 ceph-mon[36306]: Deploying daemon osd.7 on smithi100 2024-03-21T08:52:21.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:20 smithi087 ceph-mon[32385]: Deploying daemon osd.7 on smithi100 2024-03-21T08:52:21.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:20 smithi105 ceph-mon[35993]: Deploying daemon osd.7 on smithi100 2024-03-21T08:52:21.969 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:21 smithi100 ceph-mon[36306]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:22.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:21 smithi087 ceph-mon[32385]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:22.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:21 smithi105 ceph-mon[35993]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:24.155 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:23 smithi100 ceph-mon[36306]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:24.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:23 smithi087 ceph-mon[32385]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:24.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:23 smithi105 ceph-mon[35993]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:26.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:25 smithi087 ceph-mon[32385]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:26.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:26.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:26.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:26.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:25 smithi100 ceph-mon[36306]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:26.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:26.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:26.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:25 smithi105 ceph-mon[35993]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:28.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:27 smithi100 ceph-mon[36306]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:28.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:27 smithi087 ceph-mon[32385]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:27 smithi105 ceph-mon[35993]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:29.429 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 7 on host 'smithi100' 2024-03-21T08:52:29.429 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.432+0000 7f9dceffd700 1 -- 172.21.15.100:0/3153007607 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9dd00625d0 con 0x7f9db80a3030 2024-03-21T08:52:29.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9db80a3030 msgr2=0x7f9db80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:29.431 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9db80a3030 0x7f9db80a54f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9dc8005950 tx=0x7f9dc800a300 comp rx=0 tx=0).stop 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 msgr2=0x7f9dd019ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd019ef10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9dc4005910 tx=0x7f9dc40047b0 comp rx=0 tx=0).stop 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 shutdown_connections 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f9db80a3030 0x7f9db80a54f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9dd0105df0 0x7f9dd01a5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9dd0100ab0 0x7f9dd019e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 --2- 172.21.15.100:0/3153007607 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9dd0103420 0x7f9dd019ef10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 >> 172.21.15.100:0/3153007607 conn(0x7f9dd00fc6f0 msgr2=0x7f9dd0101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 shutdown_connections 2024-03-21T08:52:29.432 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:52:29.435+0000 7f9dd7d36700 1 -- 172.21.15.100:0/3153007607 wait complete. 2024-03-21T08:52:29.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.736 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:29.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:29.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:29.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:30.083 DEBUG:teuthology.orchestra.run.smithi100:osd.7> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.7.service 2024-03-21T08:52:30.085 INFO:tasks.cephadm:Deploying osd.8 on smithi105 with /dev/vg_nvme/lv_2... 2024-03-21T08:52:30.086 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-21T08:52:30.331 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:52:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:30 smithi105 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T08:52:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:30 smithi105 ceph-mon[35993]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T08:52:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:30 smithi105 ceph-mon[35993]: from='osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:52:30.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:30 smithi105 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:52:30.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-mon[36306]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T08:52:30.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-mon[36306]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T08:52:30.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:30.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-mon[36306]: from='osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:52:30.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-mon[36306]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:52:30.768 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T08:52:30.468+0000 7f50a03bd700 -1 osd.7 0 waiting for initial osdmap 2024-03-21T08:52:30.768 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 08:52:30 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T08:52:30.477+0000 7f509c1b0700 -1 osd.7 51 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:52:30.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:30 smithi087 ceph-mon[32385]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-21T08:52:30.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:30 smithi087 ceph-mon[32385]: osdmap e50: 8 total, 7 up, 8 in 2024-03-21T08:52:30.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:30.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:30 smithi087 ceph-mon[32385]: from='osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:52:30.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:30 smithi087 ceph-mon[32385]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:52:31.598 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:31 smithi100 ceph-mon[36306]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:52:31.599 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:31 smithi100 ceph-mon[36306]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T08:52:31.599 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:31.599 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:31.599 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:31 smithi100 ceph-mon[36306]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:31.642 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:31 smithi105 ceph-mon[35993]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:52:31.642 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:31 smithi105 ceph-mon[35993]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T08:52:31.643 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:31.643 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:31.643 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:31 smithi105 ceph-mon[35993]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:31.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:31 smithi087 ceph-mon[32385]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:52:31.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:31 smithi087 ceph-mon[32385]: osdmap e51: 8 total, 7 up, 8 in 2024-03-21T08:52:31.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:31.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:31.772 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:31 smithi087 ceph-mon[32385]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-03-21T08:52:32.202 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-21T08:52:32.202 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-21T08:52:32.203 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-21T08:52:32.203 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-21T08:52:32.203 INFO:teuthology.orchestra.run.smithi105.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112441 s, 933 MB/s 2024-03-21T08:52:32.203 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 2024-03-21T08:52:32.203 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-21T08:52:32.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:32 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:52:32.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:32 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:52:32.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:32 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:32.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:32 smithi105 ceph-mon[35993]: osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] boot 2024-03-21T08:52:32.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:32 smithi105 ceph-mon[35993]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T08:52:32.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:32 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:32.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:32 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:52:32.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:32 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:52:32.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:32.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:32 smithi087 ceph-mon[32385]: osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] boot 2024-03-21T08:52:32.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:32 smithi087 ceph-mon[32385]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T08:52:32.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:32 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:32.761 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:32 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:52:32.761 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:32 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:52:32.761 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:32.761 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:32 smithi100 ceph-mon[36306]: osd.7 [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] boot 2024-03-21T08:52:32.761 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:32 smithi100 ceph-mon[36306]: osdmap e52: 8 total, 8 up, 8 in 2024-03-21T08:52:32.761 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:32 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-21T08:52:32.911 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-03-21T08:52:33.154 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.584 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.585 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.585 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:33.585 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:33.585 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:33.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: osdmap e53: 8 total, 8 up, 8 in 2024-03-21T08:52:33.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: pgmap v180: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:33.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:33.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:33.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:33.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:33.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:34.672 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:34 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi100 2024-03-21T08:52:34.673 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:34 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi100 to 5693M 2024-03-21T08:52:34.673 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:34 smithi105 ceph-mon[35993]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T08:52:34.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:34 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi100 2024-03-21T08:52:34.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:34 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi100 to 5693M 2024-03-21T08:52:34.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:34 smithi100 ceph-mon[36306]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T08:52:34.902 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 -- 172.21.15.105:0/1527803321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 msgr2=0x7f4824105040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 --2- 172.21.15.105:0/1527803321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f4824105040 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4818009990 tx=0x7f4818009ca0 comp rx=0 tx=0).stop 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 -- 172.21.15.105:0/1527803321 shutdown_connections 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 --2- 172.21.15.105:0/1527803321 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824107eb0 0x7f482410a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 --2- 172.21.15.105:0/1527803321 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f4824105040 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 --2- 172.21.15.105:0/1527803321 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4824105580 0x7f4824107970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 -- 172.21.15.105:0/1527803321 >> 172.21.15.105:0/1527803321 conn(0x7f48240fc680 msgr2=0x7f48240feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 -- 172.21.15.105:0/1527803321 shutdown_connections 2024-03-21T08:52:34.903 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.906+0000 7f4829eb0700 1 -- 172.21.15.105:0/1527803321 wait complete. 2024-03-21T08:52:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.907+0000 7f4829eb0700 1 Processor -- start 2024-03-21T08:52:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.907+0000 7f4829eb0700 1 -- start start 2024-03-21T08:52:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f48241984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105580 0x7f4824198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:34.905 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4824107eb0 0x7f482419fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482410de90 con 0x7f4824107eb0 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482410dd10 con 0x7f4824105580 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482410e010 con 0x7f4824102c50 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f48241984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f48241984d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:37848/0 (socket says 172.21.15.105:37848) 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 -- 172.21.15.105:0/2750426068 learned_addr learned my addr 172.21.15.105:0/2750426068 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4822ffd700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105580 0x7f4824198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4823fff700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4824107eb0 0x7f482419fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 -- 172.21.15.105:0/2750426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105580 msgr2=0x7f4824198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105580 0x7f4824198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 -- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4824107eb0 msgr2=0x7f482419fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:34.906 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4824107eb0 0x7f482419fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 -- 172.21.15.105:0/2750426068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48241a0130 con 0x7f4824102c50 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4822ffd700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105580 0x7f4824198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f48237fe700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f48241984d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f48180047b0 tx=0x7f4818003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4818027070 con 0x7f4824102c50 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48241a03c0 con 0x7f4824102c50 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.908+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4818022470 con 0x7f4824102c50 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.909+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f481800f6c0 con 0x7f4824102c50 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.909+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48241a0890 con 0x7f4824102c50 2024-03-21T08:52:34.907 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.909+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4824068e50 con 0x7f4824102c50 2024-03-21T08:52:34.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:34 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi100 2024-03-21T08:52:34.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:34 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi100 to 5693M 2024-03-21T08:52:34.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:34 smithi087 ceph-mon[32385]: osdmap e54: 8 total, 8 up, 8 in 2024-03-21T08:52:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.910+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f481802a070 con 0x7f4824102c50 2024-03-21T08:52:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.911+0000 7f4820ff9700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f480c0a2fb0 0x7f480c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:52:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.911+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 5397+0+0 (secure 0 0 0) 0x7f48180cf9d0 con 0x7f4824102c50 2024-03-21T08:52:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.911+0000 7f4822ffd700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f480c0a2fb0 0x7f480c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:52:34.909 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.912+0000 7f4822ffd700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f480c0a2fb0 0x7f480c0a5470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4814007900 tx=0x7f4814008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:52:34.911 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:34.915+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f481805e740 con 0x7f4824102c50 2024-03-21T08:52:35.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:35.119+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f4824062690 con 0x7f480c0a2fb0 2024-03-21T08:52:35.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:35 smithi100 ceph-mon[36306]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T08:52:35.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:35 smithi100 ceph-mon[36306]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:35.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:52:35.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:52:35.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:35.858 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:35 smithi105 ceph-mon[35993]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T08:52:35.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:35 smithi105 ceph-mon[35993]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:35.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:52:35.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:52:35.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:35.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:35 smithi087 ceph-mon[32385]: osdmap e55: 8 total, 8 up, 8 in 2024-03-21T08:52:35.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:35 smithi087 ceph-mon[32385]: pgmap v183: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:35.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:52:35.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:52:35.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:36.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:36 smithi105 ceph-mon[35993]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:52:36.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:36 smithi105 ceph-mon[35993]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T08:52:36.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:36 smithi100 ceph-mon[36306]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:52:36.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:36 smithi100 ceph-mon[36306]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T08:52:36.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:36 smithi087 ceph-mon[32385]: from='client.24286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:52:36.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:36 smithi087 ceph-mon[32385]: osdmap e56: 8 total, 8 up, 8 in 2024-03-21T08:52:37.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:37 smithi100 ceph-mon[36306]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:37.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:37 smithi087 ceph-mon[32385]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:37.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:37 smithi105 ceph-mon[35993]: pgmap v185: 1 pgs: 1 remapped; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:38.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:38 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/1451993511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]: dispatch 2024-03-21T08:52:38.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:38 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]: dispatch 2024-03-21T08:52:38.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:38 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]': finished 2024-03-21T08:52:38.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:38 smithi100 ceph-mon[36306]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T08:52:38.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:38.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:38 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/1451993511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]: dispatch 2024-03-21T08:52:38.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:38 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]: dispatch 2024-03-21T08:52:38.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:38 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]': finished 2024-03-21T08:52:38.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:38 smithi087 ceph-mon[32385]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T08:52:38.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:38.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:38 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/1451993511' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]: dispatch 2024-03-21T08:52:38.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:38 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]: dispatch 2024-03-21T08:52:38.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:38 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd46697a-3872-410d-b1ab-63fa68278d79"}]': finished 2024-03-21T08:52:38.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:38 smithi105 ceph-mon[35993]: osdmap e57: 9 total, 8 up, 9 in 2024-03-21T08:52:38.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:39.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:39 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/1064759245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:52:39.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:39 smithi100 ceph-mon[36306]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:39.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:39 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/1064759245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:52:39.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:39 smithi105 ceph-mon[35993]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:39.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:39 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/1064759245' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:52:39.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:39 smithi087 ceph-mon[32385]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:41 smithi087 ceph-mon[32385]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:42.216 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:41 smithi105 ceph-mon[35993]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:41 smithi100 ceph-mon[36306]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:43 smithi087 ceph-mon[32385]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:44.184 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:43 smithi105 ceph-mon[35993]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:43 smithi100 ceph-mon[36306]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:46.124 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:45 smithi105 ceph-mon[35993]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:46.125 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:45 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T08:52:46.125 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:45 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:46.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:45 smithi087 ceph-mon[32385]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:46.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:45 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T08:52:46.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:45 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:46.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:45 smithi100 ceph-mon[36306]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:46.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:45 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-03-21T08:52:46.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:45 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:47.138 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:46 smithi105 ceph-mon[35993]: Deploying daemon osd.8 on smithi105 2024-03-21T08:52:47.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:46 smithi087 ceph-mon[32385]: Deploying daemon osd.8 on smithi105 2024-03-21T08:52:47.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:46 smithi100 ceph-mon[36306]: Deploying daemon osd.8 on smithi105 2024-03-21T08:52:48.140 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:47 smithi105 ceph-mon[35993]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:48.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:47 smithi087 ceph-mon[32385]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:48.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:47 smithi100 ceph-mon[36306]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:50.122 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:49 smithi105 ceph-mon[35993]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:50.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:49 smithi087 ceph-mon[32385]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:49 smithi100 ceph-mon[36306]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:51.941 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:51 smithi105 ceph-mon[35993]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:51.941 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:51.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:51.942 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:51 smithi087 ceph-mon[32385]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:51 smithi100 ceph-mon[36306]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:53.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:53 smithi105 ceph-mon[35993]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:54.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:53 smithi087 ceph-mon[32385]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:54.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:53 smithi100 ceph-mon[36306]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:54.582 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 8 on host 'smithi105' 2024-03-21T08:52:54.582 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.584+0000 7f4820ff9700 1 -- 172.21.15.105:0/2750426068 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4824062690 con 0x7f480c0a2fb0 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f480c0a2fb0 msgr2=0x7f480c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f480c0a2fb0 0x7f480c0a5470 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f4814007900 tx=0x7f4814008040 comp rx=0 tx=0).stop 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 msgr2=0x7f48241984d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f48241984d0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f48180047b0 tx=0x7f4818003680 comp rx=0 tx=0).stop 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 shutdown_connections 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f480c0a2fb0 0x7f480c0a5470 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4824107eb0 0x7f482419fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4824102c50 0x7f48241984d0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 --2- 172.21.15.105:0/2750426068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4824105580 0x7f4824198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 >> 172.21.15.105:0/2750426068 conn(0x7f48240fc680 msgr2=0x7f48241035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:52:54.584 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 shutdown_connections 2024-03-21T08:52:54.585 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:52:54.587+0000 7f4829eb0700 1 -- 172.21.15.105:0/2750426068 wait complete. 2024-03-21T08:52:55.330 DEBUG:teuthology.orchestra.run.smithi105:osd.8> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.8.service 2024-03-21T08:52:55.332 INFO:tasks.cephadm:Deploying osd.9 on smithi087 with /dev/vg_nvme/lv_1... 2024-03-21T08:52:55.333 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T08:52:55.580 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:52:55.629 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.630 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.630 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:55.630 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.630 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.630 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: from='osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:52:55.630 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:55 smithi105 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:52:55.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:55.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:55.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: from='osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:52:55.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:55 smithi087 ceph-mon[32385]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:52:56.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:56.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:56.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:56.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:56.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:56.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: from='osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:52:56.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:55 smithi100 ceph-mon[36306]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-03-21T08:52:57.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:56 smithi087 ceph-mon[32385]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T08:52:57.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:56 smithi087 ceph-mon[32385]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T08:52:57.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:56 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:57.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:56 smithi087 ceph-mon[32385]: from='osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:52:57.005 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:56 smithi087 ceph-mon[32385]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:52:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T08:52:57.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-mon[35993]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T08:52:57.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:57.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-mon[35993]: from='osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:52:57.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:52:57.235 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T08:52:56.894+0000 7fd7e6f18700 -1 osd.8 0 waiting for initial osdmap 2024-03-21T08:52:57.235 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 08:52:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T08:52:56.904+0000 7fd7e2d0b700 -1 osd.8 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:52:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:56 smithi100 ceph-mon[36306]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-03-21T08:52:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:56 smithi100 ceph-mon[36306]: osdmap e58: 9 total, 8 up, 9 in 2024-03-21T08:52:57.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:56 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:57.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:56 smithi100 ceph-mon[36306]: from='osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:52:57.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:56 smithi100 ceph-mon[36306]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:52:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T08:52:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T08:52:57.673 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10+0 records in 2024-03-21T08:52:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:10+0 records out 2024-03-21T08:52:57.673 INFO:teuthology.orchestra.run.smithi087.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113601 s, 923 MB/s 2024-03-21T08:52:57.673 INFO:teuthology.orchestra.run.smithi087.stderr:--> Zapping successful for: 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] boot 2024-03-21T08:52:57.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T08:52:57.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.003 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:58.003 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:52:58.003 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T08:52:58.004 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.004 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.004 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] boot 2024-03-21T08:52:58.004 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T08:52:58.004 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 232 MiB used, 715 GiB / 715 GiB avail 2024-03-21T08:52:58.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:52:58.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: osdmap e59: 9 total, 8 up, 9 in 2024-03-21T08:52:58.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: osd.8 [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] boot 2024-03-21T08:52:58.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: osdmap e60: 9 total, 9 up, 9 in 2024-03-21T08:52:58.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-03-21T08:52:58.340 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi087:vg_nvme/lv_1 2024-03-21T08:52:58.596 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:52:59.041 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:52:59.041 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:52:59.041 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:58 smithi087 ceph-mon[32385]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T08:52:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:52:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:52:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:58 smithi105 ceph-mon[35993]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T08:52:59.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:52:59.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:52:59.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:52:59.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:52:59.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:58 smithi100 ceph-mon[36306]: osdmap e61: 9 total, 9 up, 9 in 2024-03-21T08:53:00.036 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:59 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi105 2024-03-21T08:53:00.036 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:59 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-21T08:53:00.036 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:52:59 smithi087 ceph-mon[32385]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:59 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi105 2024-03-21T08:53:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:59 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-21T08:53:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:52:59 smithi105 ceph-mon[35993]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.247+0000 7f13daf42700 1 -- 172.21.15.87:0/2398273483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41033c0 msgr2=0x7f13d4103820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.247+0000 7f13daf42700 1 --2- 172.21.15.87:0/2398273483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41033c0 0x7f13d4103820 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f13c00099c0 tx=0x7f13c0009cd0 comp rx=0 tx=0).stop 2024-03-21T08:53:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.247+0000 7f13daf42700 1 -- 172.21.15.87:0/2398273483 shutdown_connections 2024-03-21T08:53:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 --2- 172.21.15.87:0/2398273483 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13d4103d60 0x7f13d410a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 --2- 172.21.15.87:0/2398273483 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d41021c0 0x7f13d41025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 --2- 172.21.15.87:0/2398273483 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41033c0 0x7f13d4103820 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- 172.21.15.87:0/2398273483 >> 172.21.15.87:0/2398273483 conn(0x7f13d40fd7a0 msgr2=0x7f13d40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- 172.21.15.87:0/2398273483 shutdown_connections 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- 172.21.15.87:0/2398273483 wait complete. 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 Processor -- start 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- start start 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 0x7f13d4075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13d41033c0 0x7f13d4074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d4103d60 0x7f13d4074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d410dd80 con 0x7f13d41021c0 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d410dc00 con 0x7f13d41033c0 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13daf42700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d410df00 con 0x7f13d4103d60 2024-03-21T08:53:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13d8cde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 0x7f13d4075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13d8cde700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 0x7f13d4075f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38390/0 (socket says 172.21.15.87:38390) 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.248+0000 7f13d8cde700 1 -- 172.21.15.87:0/693226840 learned_addr learned my addr 172.21.15.87:0/693226840 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d3fff700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13d41033c0 0x7f13d4074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d94df700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d4103d60 0x7f13d4074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d8cde700 1 -- 172.21.15.87:0/693226840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d4103d60 msgr2=0x7f13d4074ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d8cde700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d4103d60 0x7f13d4074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d8cde700 1 -- 172.21.15.87:0/693226840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13d41033c0 msgr2=0x7f13d4074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d8cde700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13d41033c0 0x7f13d4074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d8cde700 1 -- 172.21.15.87:0/693226840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d4075370 con 0x7f13d41021c0 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d8cde700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 0x7f13d4075f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f13c800d980 tx=0x7f13c800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8017030 con 0x7f13d41021c0 2024-03-21T08:53:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d41a2560 con 0x7f13d41021c0 2024-03-21T08:53:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f13c8004500 con 0x7f13d41021c0 2024-03-21T08:53:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13c8003b20 con 0x7f13d41021c0 2024-03-21T08:53:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.249+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d41a2aa0 con 0x7f13d41021c0 2024-03-21T08:53:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.250+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d4068e50 con 0x7f13d41021c0 2024-03-21T08:53:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.251+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f13c801a070 con 0x7f13d41021c0 2024-03-21T08:53:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.252+0000 7f13d1ffb700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13c40a2fb0 0x7f13c40a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.252+0000 7f13d3fff700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13c40a2fb0 0x7f13c40a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.252+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f13c80ce490 con 0x7f13d41021c0 2024-03-21T08:53:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.252+0000 7f13d3fff700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13c40a2fb0 0x7f13c40a5470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:53:00.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.256+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f13c8009940 con 0x7f13d41021c0 2024-03-21T08:53:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:59 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi105 2024-03-21T08:53:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:59 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi105 to 5693M 2024-03-21T08:53:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:52:59 smithi100 ceph-mon[36306]: pgmap v200: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:00.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:00.461+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f13d40625d0 con 0x7f13c40a2fb0 2024-03-21T08:53:01.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:00 smithi087 ceph-mon[32385]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T08:53:01.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:01.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:01.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:01.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:00 smithi105 ceph-mon[35993]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T08:53:01.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:01.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:01.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:01.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:00 smithi100 ceph-mon[36306]: osdmap e62: 9 total, 9 up, 9 in 2024-03-21T08:53:01.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:01.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:01.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:02.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:01 smithi087 ceph-mon[32385]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:02.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:01 smithi087 ceph-mon[32385]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:02.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:01 smithi087 ceph-mon[32385]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T08:53:02.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:01 smithi105 ceph-mon[35993]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:02.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:01 smithi105 ceph-mon[35993]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:02.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:01 smithi105 ceph-mon[35993]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T08:53:02.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:01 smithi100 ceph-mon[36306]: from='client.14424 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi087:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:02.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:01 smithi100 ceph-mon[36306]: pgmap v203: 1 pgs: 1 peering; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:02.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:01 smithi100 ceph-mon[36306]: osdmap e63: 9 total, 9 up, 9 in 2024-03-21T08:53:03.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:02 smithi105 ceph-mon[35993]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T08:53:03.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:02 smithi100 ceph-mon[36306]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T08:53:03.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:02 smithi087 ceph-mon[32385]: osdmap e64: 9 total, 9 up, 9 in 2024-03-21T08:53:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:03 smithi087 ceph-mon[32385]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:53:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1155923071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2aee04df-d0a2-48c3-89b5-eb79810633c0"}]: dispatch 2024-03-21T08:53:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1155923071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2aee04df-d0a2-48c3-89b5-eb79810633c0"}]': finished 2024-03-21T08:53:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:03 smithi087 ceph-mon[32385]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T08:53:04.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:04.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:03 smithi105 ceph-mon[35993]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:53:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1155923071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2aee04df-d0a2-48c3-89b5-eb79810633c0"}]: dispatch 2024-03-21T08:53:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1155923071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2aee04df-d0a2-48c3-89b5-eb79810633c0"}]': finished 2024-03-21T08:53:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:03 smithi105 ceph-mon[35993]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T08:53:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:03 smithi100 ceph-mon[36306]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 259 MiB used, 804 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:53:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1155923071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2aee04df-d0a2-48c3-89b5-eb79810633c0"}]: dispatch 2024-03-21T08:53:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1155923071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2aee04df-d0a2-48c3-89b5-eb79810633c0"}]': finished 2024-03-21T08:53:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:03 smithi100 ceph-mon[36306]: osdmap e65: 10 total, 9 up, 10 in 2024-03-21T08:53:04.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:05.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:04 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/622441876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:05.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:04 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/622441876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:05.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:04 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/622441876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:06.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:05 smithi105 ceph-mon[35993]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-21T08:53:06.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:05 smithi100 ceph-mon[36306]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-21T08:53:06.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:05 smithi087 ceph-mon[32385]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-21T08:53:08.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:07 smithi087 ceph-mon[32385]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:53:08.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:07 smithi105 ceph-mon[35993]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:53:08.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:07 smithi100 ceph-mon[36306]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-21T08:53:10.055 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:09 smithi087 ceph-mon[32385]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T08:53:10.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:09 smithi105 ceph-mon[35993]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T08:53:10.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:09 smithi100 ceph-mon[36306]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-03-21T08:53:12.064 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:11 smithi087 ceph-mon[32385]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T08:53:12.064 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T08:53:12.064 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:12.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:11 smithi105 ceph-mon[35993]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T08:53:12.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T08:53:12.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:11 smithi100 ceph-mon[36306]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-03-21T08:53:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-03-21T08:53:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:13.143 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:12 smithi087 ceph-mon[32385]: Deploying daemon osd.9 on smithi087 2024-03-21T08:53:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:12 smithi105 ceph-mon[35993]: Deploying daemon osd.9 on smithi087 2024-03-21T08:53:13.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:12 smithi100 ceph-mon[36306]: Deploying daemon osd.9 on smithi087 2024-03-21T08:53:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:13 smithi105 ceph-mon[35993]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:13 smithi100 ceph-mon[36306]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:14.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:13 smithi087 ceph-mon[32385]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:16.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:15 smithi105 ceph-mon[35993]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:16.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:15 smithi100 ceph-mon[36306]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:16.363 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:15 smithi087 ceph-mon[32385]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:18.063 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:17 smithi087 ceph-mon[32385]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:18.063 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:18.063 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:18.064 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:18.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:17 smithi105 ceph-mon[35993]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:18.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:18.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:18.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:18.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:17 smithi100 ceph-mon[36306]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:18.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:18.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:18.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:20.230 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:19 smithi087 ceph-mon[32385]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:20.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:19 smithi105 ceph-mon[35993]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:20.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:19 smithi100 ceph-mon[36306]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:21.420 INFO:teuthology.orchestra.run.smithi087.stdout:Created osd(s) 9 on host 'smithi087' 2024-03-21T08:53:21.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.423+0000 7f13d1ffb700 1 -- 172.21.15.87:0/693226840 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f13d40625d0 con 0x7f13c40a2fb0 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13c40a2fb0 msgr2=0x7f13c40a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13c40a2fb0 0x7f13c40a5470 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f13c0000c00 tx=0x7f13c0005fd0 comp rx=0 tx=0).stop 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 msgr2=0x7f13d4075f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 0x7f13d4075f30 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f13c800d980 tx=0x7f13c800dd40 comp rx=0 tx=0).stop 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 shutdown_connections 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13c40a2fb0 0x7f13c40a5470 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13d4103d60 0x7f13d4074ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13d41021c0 0x7f13d4075f30 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 --2- 172.21.15.87:0/693226840 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13d41033c0 0x7f13d4074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:21.422 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 >> 172.21.15.87:0/693226840 conn(0x7f13d40fd7a0 msgr2=0x7f13d4106740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 shutdown_connections 2024-03-21T08:53:21.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:53:21.426+0000 7f13daf42700 1 -- 172.21.15.87:0/693226840 wait complete. 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T08:53:21.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:21 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:21.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:21.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:21.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:21.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T08:53:21.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:21 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:21.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:21.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:21.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:21.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-03-21T08:53:21.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:21.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:21 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:22.059 DEBUG:teuthology.orchestra.run.smithi087:osd.9> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.9.service 2024-03-21T08:53:22.062 INFO:tasks.cephadm:Deploying osd.10 on smithi100 with /dev/vg_nvme/lv_1... 2024-03-21T08:53:22.062 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T08:53:22.308 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:53:22.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:22 smithi087 ceph-mon[32385]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T08:53:22.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:22 smithi087 ceph-mon[32385]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T08:53:22.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:22 smithi087 ceph-mon[32385]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:53:22.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:22 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:22.682 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:22 smithi100 ceph-mon[36306]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T08:53:22.682 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:22 smithi100 ceph-mon[36306]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T08:53:22.682 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:22 smithi100 ceph-mon[36306]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:53:22.682 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:22 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:22.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:22 smithi105 ceph-mon[35993]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-03-21T08:53:22.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:22 smithi105 ceph-mon[35993]: osdmap e66: 10 total, 9 up, 10 in 2024-03-21T08:53:22.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:22 smithi105 ceph-mon[35993]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]: dispatch 2024-03-21T08:53:22.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:22 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.157 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 08:53:22 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T08:53:22.922+0000 7fcb9ce3c700 -1 osd.9 0 waiting for initial osdmap 2024-03-21T08:53:23.157 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 08:53:22 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T08:53:22.933+0000 7fcb94c27700 -1 osd.9 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:53:23.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:23 smithi087 ceph-mon[32385]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:53:23.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:23 smithi087 ceph-mon[32385]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T08:53:23.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:23 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.532 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:23 smithi087 ceph-mon[32385]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:23.533 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:23 smithi087 ceph-mon[32385]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' 2024-03-21T08:53:23.711 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:23 smithi100 ceph-mon[36306]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:53:23.711 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:23 smithi100 ceph-mon[36306]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T08:53:23.711 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.711 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:23 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.711 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:23 smithi100 ceph-mon[36306]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:23.711 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:23 smithi100 ceph-mon[36306]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' 2024-03-21T08:53:23.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:23 smithi105 ceph-mon[35993]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi087", "root=default"]}]': finished 2024-03-21T08:53:23.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:23 smithi105 ceph-mon[35993]: osdmap e67: 10 total, 9 up, 10 in 2024-03-21T08:53:23.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:23 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:23.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:23 smithi105 ceph-mon[35993]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 260 MiB used, 804 GiB / 805 GiB avail 2024-03-21T08:53:23.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:23 smithi105 ceph-mon[35993]: from='osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550]' entity='osd.9' 2024-03-21T08:53:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T08:53:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T08:53:24.465 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10+0 records in 2024-03-21T08:53:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:10+0 records out 2024-03-21T08:53:24.465 INFO:teuthology.orchestra.run.smithi100.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130946 s, 801 MB/s 2024-03-21T08:53:24.465 INFO:teuthology.orchestra.run.smithi100.stderr:--> Zapping successful for: 2024-03-21T08:53:24.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:24 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:53:24.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:24 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:53:24.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:24 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:24.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:24 smithi100 ceph-mon[36306]: osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] boot 2024-03-21T08:53:24.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:24 smithi100 ceph-mon[36306]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T08:53:24.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:24 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:24.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:24 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:53:24.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:24 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:53:24.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:24 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:24.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:24 smithi087 ceph-mon[32385]: osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] boot 2024-03-21T08:53:24.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:24 smithi087 ceph-mon[32385]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T08:53:24.778 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:24 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:24.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:24 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:53:24.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:24 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:53:24.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:24 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:24.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:24 smithi105 ceph-mon[35993]: osd.9 [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] boot 2024-03-21T08:53:24.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:24 smithi105 ceph-mon[35993]: osdmap e68: 10 total, 10 up, 10 in 2024-03-21T08:53:24.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:24 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-03-21T08:53:25.128 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi100:vg_nvme/lv_1 2024-03-21T08:53:25.370 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T08:53:26.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:26.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T08:53:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:26.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: osdmap e69: 10 total, 10 up, 10 in 2024-03-21T08:53:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:27.160 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 -- 172.21.15.100:0/3912835875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92741021b0 msgr2=0x7f92741025d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3912835875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92741021b0 0x7f92741025d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f92640099c0 tx=0x7f9264009cd0 comp rx=0 tx=0).stop 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 -- 172.21.15.100:0/3912835875 shutdown_connections 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3912835875 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f92741021b0 0x7f92741025d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3912835875 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9274103d50 0x7f927410a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3912835875 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741033b0 0x7f9274103810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 -- 172.21.15.100:0/3912835875 >> 172.21.15.100:0/3912835875 conn(0x7f92740fd770 msgr2=0x7f92740ffb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 -- 172.21.15.100:0/3912835875 shutdown_connections 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.161+0000 7f9279d4e700 1 -- 172.21.15.100:0/3912835875 wait complete. 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 Processor -- start 2024-03-21T08:53:27.161 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 -- start start 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741021b0 0x7f9274075f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741033b0 0x7f9274074580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 0x7f9274074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927410dec0 con 0x7f92741021b0 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927410dd40 con 0x7f9274103d50 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.162+0000 7f9279d4e700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927410e040 con 0x7f92741033b0 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 0x7f9274074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 0x7f9274074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.100:52268/0 (socket says 172.21.15.100:52268) 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 -- 172.21.15.100:0/3777088295 learned_addr learned my addr 172.21.15.100:0/3777088295 (peer_addr_for_me v2:172.21.15.100:0/0) 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f92737fe700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741021b0 0x7f9274075f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:27.162 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9272ffd700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741033b0 0x7f9274074580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 -- 172.21.15.100:0/3777088295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741033b0 msgr2=0x7f9274074580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741033b0 0x7f9274074580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 -- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741021b0 msgr2=0x7f9274075f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741021b0 0x7f9274075f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 -- 172.21.15.100:0/3777088295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9274075370 con 0x7f9274103d50 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9272ffd700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741033b0 0x7f9274074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f92737fe700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741021b0 0x7f9274075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9273fff700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 0x7f9274074ac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f926c00b970 tx=0x7f926c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c019070 con 0x7f9274103d50 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92741a2720 con 0x7f9274103d50 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f926c014440 con 0x7f9274103d50 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c0075d0 con 0x7f9274103d50 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.163+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92741a2cb0 con 0x7f9274103d50 2024-03-21T08:53:27.163 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.164+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9274068e50 con 0x7f9274103d50 2024-03-21T08:53:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.165+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f926c01c070 con 0x7f9274103d50 2024-03-21T08:53:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.166+0000 7f9270ff9700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f925c0a2fb0 0x7f925c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:27.164 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.166+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6261+0+0 (secure 0 0 0) 0x7f926c0ce040 con 0x7f9274103d50 2024-03-21T08:53:27.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.166+0000 7f92737fe700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f925c0a2fb0 0x7f925c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:27.165 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.167+0000 7f92737fe700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f925c0a2fb0 0x7f925c0a5470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f926400b580 tx=0x7f9264005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:53:27.166 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.171+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f926c09b670 con 0x7f9274103d50 2024-03-21T08:53:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:26 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi087 2024-03-21T08:53:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:26 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi087 to 4272M 2024-03-21T08:53:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:26 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi087 2024-03-21T08:53:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:26 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi087 to 4272M 2024-03-21T08:53:27.374 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:27.378+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f92740625d0 con 0x7f925c0a2fb0 2024-03-21T08:53:27.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:26 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi087 2024-03-21T08:53:27.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:26 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi087 to 4272M 2024-03-21T08:53:28.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:27 smithi100 ceph-mon[36306]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:28.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:28.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:28.217 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:27 smithi105 ceph-mon[35993]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:27 smithi087 ceph-mon[32385]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:29.189 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:28 smithi100 ceph-mon[36306]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:29.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:28 smithi105 ceph-mon[35993]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:29.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:28 smithi087 ceph-mon[32385]: from='client.24320 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi100:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:29 smithi105 ceph-mon[35993]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:30.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:29 smithi100 ceph-mon[36306]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:30.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:29 smithi087 ceph-mon[32385]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:31.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:30 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/597438104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]: dispatch 2024-03-21T08:53:31.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:30 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]: dispatch 2024-03-21T08:53:31.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:30 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]': finished 2024-03-21T08:53:31.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:30 smithi105 ceph-mon[35993]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T08:53:31.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:31.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:30 smithi105 ceph-mon[35993]: from='client.? 172.21.15.100:0/3740779978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:31.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:30 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/597438104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]: dispatch 2024-03-21T08:53:31.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:30 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]: dispatch 2024-03-21T08:53:31.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:30 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]': finished 2024-03-21T08:53:31.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:30 smithi100 ceph-mon[36306]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T08:53:31.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:31.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:30 smithi100 ceph-mon[36306]: from='client.? 172.21.15.100:0/3740779978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:30 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/597438104' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]: dispatch 2024-03-21T08:53:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:30 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]: dispatch 2024-03-21T08:53:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:30 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cbfb1970-a260-4470-b56b-eff1363dd1b5"}]': finished 2024-03-21T08:53:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:30 smithi087 ceph-mon[32385]: osdmap e70: 11 total, 10 up, 11 in 2024-03-21T08:53:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:30 smithi087 ceph-mon[32385]: from='client.? 172.21.15.100:0/3740779978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:32.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:31 smithi105 ceph-mon[35993]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:32.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:31 smithi100 ceph-mon[36306]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:32.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:31 smithi087 ceph-mon[32385]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:34.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:33 smithi105 ceph-mon[35993]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:34.251 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:33 smithi087 ceph-mon[32385]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:34.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:33 smithi100 ceph-mon[36306]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:35 smithi105 ceph-mon[35993]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:35 smithi100 ceph-mon[36306]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:36.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:35 smithi087 ceph-mon[32385]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:38.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:37 smithi100 ceph-mon[36306]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:38.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T08:53:38.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:38.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:37 smithi105 ceph-mon[35993]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:38.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T08:53:38.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:38.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:37 smithi087 ceph-mon[32385]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:38.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-03-21T08:53:38.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:39.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:38 smithi105 ceph-mon[35993]: Deploying daemon osd.10 on smithi100 2024-03-21T08:53:39.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:38 smithi100 ceph-mon[36306]: Deploying daemon osd.10 on smithi100 2024-03-21T08:53:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:38 smithi087 ceph-mon[32385]: Deploying daemon osd.10 on smithi100 2024-03-21T08:53:40.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:39 smithi100 ceph-mon[36306]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:40.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:39 smithi105 ceph-mon[35993]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:40.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:39 smithi087 ceph-mon[32385]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:42.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:41 smithi100 ceph-mon[36306]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:42.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:41 smithi105 ceph-mon[35993]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:42.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:41 smithi087 ceph-mon[32385]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:43 smithi105 ceph-mon[35993]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:44.234 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:43 smithi100 ceph-mon[36306]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:44.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:44.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:44.235 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:44.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:43 smithi087 ceph-mon[32385]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:44.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:44.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:44.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:46.221 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:45 smithi100 ceph-mon[36306]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:46.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:45 smithi105 ceph-mon[35993]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:46.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:45 smithi087 ceph-mon[32385]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:47.542 INFO:teuthology.orchestra.run.smithi100.stdout:Created osd(s) 10 on host 'smithi100' 2024-03-21T08:53:47.542 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.546+0000 7f9270ff9700 1 -- 172.21.15.100:0/3777088295 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f92740625d0 con 0x7f925c0a2fb0 2024-03-21T08:53:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f925c0a2fb0 msgr2=0x7f925c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f925c0a2fb0 0x7f925c0a5470 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f926400b580 tx=0x7f9264005fb0 comp rx=0 tx=0).stop 2024-03-21T08:53:47.544 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 msgr2=0x7f9274074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 0x7f9274074ac0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f926c00b970 tx=0x7f926c00bd30 comp rx=0 tx=0).stop 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 shutdown_connections 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f925c0a2fb0 0x7f925c0a5470 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9274103d50 0x7f9274074ac0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f92741021b0 0x7f9274075f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 --2- 172.21.15.100:0/3777088295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f92741033b0 0x7f9274074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 >> 172.21.15.100:0/3777088295 conn(0x7f92740fd770 msgr2=0x7f9274106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 shutdown_connections 2024-03-21T08:53:47.545 INFO:teuthology.orchestra.run.smithi100.stderr:2024-03-21T08:53:47.548+0000 7f9279d4e700 1 -- 172.21.15.100:0/3777088295 wait complete. 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:53:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:53:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:47.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:53:47.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:53:47.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:53:47.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-03-21T08:53:47.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:47.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:53:48.182 DEBUG:teuthology.orchestra.run.smithi100:osd.10> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.10.service 2024-03-21T08:53:48.184 INFO:tasks.cephadm:Deploying osd.11 on smithi105 with /dev/vg_nvme/lv_1... 2024-03-21T08:53:48.184 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-21T08:53:48.430 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:53:48.687 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-mon[36306]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T08:53:48.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-mon[36306]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T08:53:48.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:48.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-mon[36306]: from='osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:53:48.688 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-mon[36306]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:53:48.688 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T08:53:48.649+0000 7fc0df579700 -1 osd.10 0 waiting for initial osdmap 2024-03-21T08:53:48.688 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 08:53:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T08:53:48.660+0000 7fc0db36c700 -1 osd.10 72 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:53:48.875 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:48 smithi105 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T08:53:48.875 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:48 smithi105 ceph-mon[35993]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T08:53:48.875 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:48.875 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:48 smithi105 ceph-mon[35993]: from='osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:53:48.875 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:48 smithi105 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:53:48.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:48 smithi087 ceph-mon[32385]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-03-21T08:53:48.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:48 smithi087 ceph-mon[32385]: osdmap e71: 11 total, 10 up, 11 in 2024-03-21T08:53:48.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:48.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:48 smithi087 ceph-mon[32385]: from='osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:53:48.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:48 smithi087 ceph-mon[32385]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]: dispatch 2024-03-21T08:53:49.746 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:53:49.746 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T08:53:49.746 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.746 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.746 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:49.746 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] boot 2024-03-21T08:53:49.747 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T08:53:49.747 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] boot 2024-03-21T08:53:49.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T08:53:49.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.916 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi100", "root=default"]}]': finished 2024-03-21T08:53:49.916 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: osdmap e72: 11 total, 10 up, 11 in 2024-03-21T08:53:49.916 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.916 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:49.916 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 291 MiB used, 894 GiB / 894 GiB avail 2024-03-21T08:53:49.916 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: osd.10 [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] boot 2024-03-21T08:53:49.917 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: osdmap e73: 11 total, 11 up, 11 in 2024-03-21T08:53:49.917 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-03-21T08:53:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-21T08:53:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-21T08:53:50.505 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-03-21T08:53:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-03-21T08:53:50.505 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113396 s, 925 MB/s 2024-03-21T08:53:50.505 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-03-21T08:53:51.137 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:50 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:53:51.138 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:50 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:53:51.138 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:50 smithi105 ceph-mon[35993]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T08:53:51.164 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-03-21T08:53:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:50 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:53:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:50 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:53:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:50 smithi100 ceph-mon[36306]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T08:53:51.402 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:53:51.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:50 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:53:51.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:50 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:53:51.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:50 smithi087 ceph-mon[32385]: osdmap e74: 11 total, 11 up, 11 in 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:52.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:52.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:51 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.224 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:52.225 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:51 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:53:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: osdmap e75: 11 total, 11 up, 11 in 2024-03-21T08:53:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:53:52.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:51 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:53:53.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 -- 172.21.15.105:0/395951876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014107f50 msgr2=0x7f401410a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:53.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 --2- 172.21.15.105:0/395951876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014107f50 0x7f401410a340 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f400c00b0a0 tx=0x7f400c00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:53:53.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 -- 172.21.15.105:0/395951876 shutdown_connections 2024-03-21T08:53:53.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 --2- 172.21.15.105:0/395951876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014107f50 0x7f401410a340 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:53.138 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 --2- 172.21.15.105:0/395951876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4014102cf0 0x7f40141050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 --2- 172.21.15.105:0/395951876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014105620 0x7f4014107a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.139+0000 7f401bbc9700 1 -- 172.21.15.105:0/395951876 >> 172.21.15.105:0/395951876 conn(0x7f40140fc680 msgr2=0x7f40140feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 -- 172.21.15.105:0/395951876 shutdown_connections 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 -- 172.21.15.105:0/395951876 wait complete. 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 Processor -- start 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 -- start start 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 0x7f4014102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014105620 0x7f4014100ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4014107f50 0x7f40141011e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401410dd70 con 0x7f4014107f50 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401410dbf0 con 0x7f4014105620 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401bbc9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f401410def0 con 0x7f4014102cf0 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f4019965700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 0x7f4014102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:53.139 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f4019965700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 0x7f4014102650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:46398/0 (socket says 172.21.15.105:46398) 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f4019965700 1 -- 172.21.15.105:0/2325944968 learned_addr learned my addr 172.21.15.105:0/2325944968 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.140+0000 7f401a166700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4014107f50 0x7f40141011e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019164700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014105620 0x7f4014100ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019965700 1 -- 172.21.15.105:0/2325944968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014105620 msgr2=0x7f4014100ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019965700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014105620 0x7f4014100ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019965700 1 -- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4014107f50 msgr2=0x7f40141011e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019965700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4014107f50 0x7f40141011e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019965700 1 -- 172.21.15.105:0/2325944968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4014101a90 con 0x7f4014102cf0 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f4019965700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 0x7f4014102650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f401000d980 tx=0x7f401000dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4010016070 con 0x7f4014102cf0 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4010004d10 con 0x7f4014102cf0 2024-03-21T08:53:53.140 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40100055f0 con 0x7f4014102cf0 2024-03-21T08:53:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40141a24c0 con 0x7f4014102cf0 2024-03-21T08:53:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.141+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40141a2a00 con 0x7f4014102cf0 2024-03-21T08:53:53.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.142+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4014068e50 con 0x7f4014102cf0 2024-03-21T08:53:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.143+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4010004750 con 0x7f4014102cf0 2024-03-21T08:53:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.144+0000 7f400affd700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40000a3030 0x7f40000a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:53:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.144+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 6694+0+0 (secure 0 0 0) 0x7f40100cf0b0 con 0x7f4014102cf0 2024-03-21T08:53:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.144+0000 7f4019164700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40000a3030 0x7f40000a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:53:53.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.144+0000 7f4019164700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40000a3030 0x7f40000a54f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4004005950 tx=0x7f40040058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:53:53.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.147+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f401001f020 con 0x7f4014102cf0 2024-03-21T08:53:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:52 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi100 2024-03-21T08:53:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:52 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi100 to 4270M 2024-03-21T08:53:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:52 smithi105 ceph-mon[35993]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T08:53:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:52 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi100 2024-03-21T08:53:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:52 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi100 to 4270M 2024-03-21T08:53:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:52 smithi100 ceph-mon[36306]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T08:53:53.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:53:53.354+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f40140625d0 con 0x7f40000a3030 2024-03-21T08:53:53.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:52 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi100 2024-03-21T08:53:53.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:52 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi100 to 4270M 2024-03-21T08:53:53.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:52 smithi087 ceph-mon[32385]: osdmap e76: 11 total, 11 up, 11 in 2024-03-21T08:53:54.210 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:53 smithi105 ceph-mon[35993]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:53:54.210 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:53 smithi105 ceph-mon[35993]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T08:53:54.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:54.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:54.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:53 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:54.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:53 smithi100 ceph-mon[36306]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:53:54.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:53 smithi100 ceph-mon[36306]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T08:53:54.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:54.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:54.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:53 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:54.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:53 smithi087 ceph-mon[32385]: pgmap v243: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-21T08:53:54.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:53 smithi087 ceph-mon[32385]: osdmap e77: 11 total, 11 up, 11 in 2024-03-21T08:53:54.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-21T08:53:54.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-21T08:53:54.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:53 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:53:55.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:54 smithi105 ceph-mon[35993]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:55.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:54 smithi100 ceph-mon[36306]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:55.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:54 smithi087 ceph-mon[32385]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:53:56.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:55 smithi105 ceph-mon[35993]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-21T08:53:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:55 smithi100 ceph-mon[36306]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-21T08:53:56.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:55 smithi087 ceph-mon[32385]: pgmap v245: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 117 KiB/s, 0 objects/s recovering 2024-03-21T08:53:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:56 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/645233221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]: dispatch 2024-03-21T08:53:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:56 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]: dispatch 2024-03-21T08:53:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:56 smithi105 ceph-mon[35993]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]': finished 2024-03-21T08:53:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:56 smithi105 ceph-mon[35993]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T08:53:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:56 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:53:57.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:56 smithi105 ceph-mon[35993]: from='client.? 172.21.15.105:0/2589613366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:56 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/645233221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]: dispatch 2024-03-21T08:53:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:56 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]: dispatch 2024-03-21T08:53:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:56 smithi100 ceph-mon[36306]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]': finished 2024-03-21T08:53:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:56 smithi100 ceph-mon[36306]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T08:53:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:56 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:53:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:56 smithi100 ceph-mon[36306]: from='client.? 172.21.15.105:0/2589613366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:57.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:56 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/645233221' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]: dispatch 2024-03-21T08:53:57.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:56 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]: dispatch 2024-03-21T08:53:57.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:56 smithi087 ceph-mon[32385]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d12fe1b9-2770-49d6-a7ca-d6595f7041f2"}]': finished 2024-03-21T08:53:57.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:56 smithi087 ceph-mon[32385]: osdmap e78: 12 total, 11 up, 12 in 2024-03-21T08:53:57.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:56 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:53:57.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:56 smithi087 ceph-mon[32385]: from='client.? 172.21.15.105:0/2589613366' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-21T08:53:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:57 smithi105 ceph-mon[35993]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T08:53:58.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:57 smithi100 ceph-mon[36306]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T08:53:58.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:57 smithi087 ceph-mon[32385]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-03-21T08:54:00.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:53:59 smithi105 ceph-mon[35993]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-21T08:54:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:53:59 smithi100 ceph-mon[36306]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-21T08:54:00.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:53:59 smithi087 ceph-mon[32385]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail; 83 KiB/s, 0 objects/s recovering 2024-03-21T08:54:02.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:01 smithi105 ceph-mon[35993]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:02.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:01 smithi100 ceph-mon[36306]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:02.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:01 smithi087 ceph-mon[32385]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:04.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:03 smithi105 ceph-mon[35993]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:04.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T08:54:04.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:03 smithi100 ceph-mon[36306]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T08:54:04.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:04.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:03 smithi087 ceph-mon[32385]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:04.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-03-21T08:54:04.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:05.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:04 smithi105 ceph-mon[35993]: Deploying daemon osd.11 on smithi105 2024-03-21T08:54:05.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:04 smithi100 ceph-mon[36306]: Deploying daemon osd.11 on smithi105 2024-03-21T08:54:05.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:04 smithi087 ceph-mon[32385]: Deploying daemon osd.11 on smithi105 2024-03-21T08:54:06.136 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:05 smithi105 ceph-mon[35993]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:06.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:05 smithi100 ceph-mon[36306]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:06.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:05 smithi087 ceph-mon[32385]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:08.122 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:07 smithi105 ceph-mon[35993]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:08.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:07 smithi100 ceph-mon[36306]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:08.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:07 smithi087 ceph-mon[32385]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:10.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:09 smithi105 ceph-mon[35993]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:10.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:10.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:10.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:10.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:09 smithi100 ceph-mon[36306]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:10.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:10.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:10.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:10.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:09 smithi087 ceph-mon[32385]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:10.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:10.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:10.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:12.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:11 smithi105 ceph-mon[35993]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:11 smithi100 ceph-mon[36306]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:12.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:11 smithi087 ceph-mon[32385]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:13.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.727+0000 7f400affd700 1 -- 172.21.15.105:0/2325944968 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f40140625d0 con 0x7f40000a3030 2024-03-21T08:54:13.723 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 11 on host 'smithi105' 2024-03-21T08:54:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40000a3030 msgr2=0x7f40000a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:13.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40000a3030 0x7f40000a54f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4004005950 tx=0x7f40040058e0 comp rx=0 tx=0).stop 2024-03-21T08:54:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 msgr2=0x7f4014102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 0x7f4014102650 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f401000d980 tx=0x7f401000dc90 comp rx=0 tx=0).stop 2024-03-21T08:54:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 shutdown_connections 2024-03-21T08:54:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40000a3030 0x7f40000a54f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:13.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4014107f50 0x7f40141011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4014102cf0 0x7f4014102650 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 --2- 172.21.15.105:0/2325944968 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4014105620 0x7f4014100ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 >> 172.21.15.105:0/2325944968 conn(0x7f40140fc680 msgr2=0x7f4014103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 shutdown_connections 2024-03-21T08:54:13.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:13.730+0000 7f401bbc9700 1 -- 172.21.15.105:0/2325944968 wait complete. 2024-03-21T08:54:13.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:13.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:13.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:13.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:13.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:13.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:13.987 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:13.988 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:13.988 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:13.988 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:54:13.988 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:13 smithi105 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:54:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:14.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:54:14.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:13 smithi087 ceph-mon[32385]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:54:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:54:14.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:13 smithi100 ceph-mon[36306]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-03-21T08:54:14.437 DEBUG:teuthology.orchestra.run.smithi105:osd.11> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.11.service 2024-03-21T08:54:14.440 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-03-21T08:54:14.440 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd stat -f json 2024-03-21T08:54:14.689 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:54:15.955 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:15 smithi087 ceph-mon[32385]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T08:54:15.955 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:15 smithi087 ceph-mon[32385]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T08:54:15.955 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:15 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:15.955 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:15 smithi087 ceph-mon[32385]: from='osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:54:15.955 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:15 smithi087 ceph-mon[32385]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:54:15.955 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:15 smithi087 ceph-mon[32385]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:15.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T08:54:15.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-mon[35993]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T08:54:15.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:15.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-mon[35993]: from='osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:54:15.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:54:15.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-mon[35993]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:15.984 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T08:54:15.711+0000 7f18225b2700 -1 osd.11 0 waiting for initial osdmap 2024-03-21T08:54:15.985 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 08:54:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T08:54:15.727+0000 7f181e3a5700 -1 osd.11 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-03-21T08:54:16.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:15 smithi100 ceph-mon[36306]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-03-21T08:54:16.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:15 smithi100 ceph-mon[36306]: osdmap e79: 12 total, 11 up, 12 in 2024-03-21T08:54:16.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:15 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:16.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:15 smithi100 ceph-mon[36306]: from='osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:54:16.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:15 smithi100 ceph-mon[36306]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-03-21T08:54:16.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:15 smithi100 ceph-mon[36306]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 319 MiB used, 983 GiB / 983 GiB avail 2024-03-21T08:54:16.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/1751231612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c81026a0 msgr2=0x7fc1c8102ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:16.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/1751231612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c81026a0 0x7fc1c8102ac0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000b0d0 tx=0x7fc1c000b3e0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/1751231612 shutdown_connections 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/1751231612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c8101d80 0x7fc1c8109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/1751231612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c81026a0 0x7fc1c8102ac0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/1751231612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 0x7fc1c8101840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/1751231612 >> 172.21.15.87:0/1751231612 conn(0x7fc1c80fbe30 msgr2=0x7fc1c80fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/1751231612 shutdown_connections 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.504+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/1751231612 wait complete. 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 Processor -- start 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 -- start start 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 0x7fc1c8198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 0x7fc1c8198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:16.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c81026a0 0x7fc1c819f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c810cd00 con 0x7fc1c8101d80 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c810cb80 con 0x7fc1c81013e0 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1cd3cb700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c810ce80 con 0x7fc1c81026a0 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1c67fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 0x7fc1c8198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1c67fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 0x7fc1c8198990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50464/0 (socket says 172.21.15.87:50464) 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.505+0000 7fc1c67fc700 1 -- 172.21.15.87:0/4010069663 learned_addr learned my addr 172.21.15.87:0/4010069663 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c6ffd700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 0x7fc1c8198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c77fe700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c81026a0 0x7fc1c819f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c67fc700 1 -- 172.21.15.87:0/4010069663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c81026a0 msgr2=0x7fc1c819f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c67fc700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c81026a0 0x7fc1c819f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c67fc700 1 -- 172.21.15.87:0/4010069663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 msgr2=0x7fc1c8198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:16.504 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c67fc700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 0x7fc1c8198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c67fc700 1 -- 172.21.15.87:0/4010069663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c81a0060 con 0x7fc1c8101d80 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c6ffd700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 0x7fc1c8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c77fe700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c81026a0 0x7fc1c819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1c67fc700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 0x7fc1c8198990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c950 tx=0x7fc1bc00cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc007ac0 con 0x7fc1c8101d80 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c81a0350 con 0x7fc1c8101d80 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc1bc004d10 con 0x7fc1c8101d80 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1bc005680 con 0x7fc1c8101d80 2024-03-21T08:54:16.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.506+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c81a08e0 con 0x7fc1c8101d80 2024-03-21T08:54:16.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.507+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c8068e50 con 0x7fc1c8101d80 2024-03-21T08:54:16.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.509+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc1bc004750 con 0x7fc1c8101d80 2024-03-21T08:54:16.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.509+0000 7fc1affff700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1b00a2fb0 0x7fc1b00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:16.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.509+0000 7fc1c6ffd700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1b00a2fb0 0x7fc1b00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:16.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.509+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fc1bc0cee30 con 0x7fc1c8101d80 2024-03-21T08:54:16.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.510+0000 7fc1c6ffd700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1b00a2fb0 0x7fc1b00a5470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000f80 tx=0x7fc1c000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:16.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.513+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc1bc014030 con 0x7fc1c8101d80 2024-03-21T08:54:16.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.708+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc1c81a0d10 con 0x7fc1c8101d80 2024-03-21T08:54:16.704 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.709+0000 7fc1affff700 1 -- 172.21.15.87:0/4010069663 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7fc1bc09c220 con 0x7fc1c8101d80 2024-03-21T08:54:16.705 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1b00a2fb0 msgr2=0x7fc1b00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1b00a2fb0 0x7fc1b00a5470 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc1c0000f80 tx=0x7fc1c000b040 comp rx=0 tx=0).stop 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 msgr2=0x7fc1c8198990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 0x7fc1c8198990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc00c950 tx=0x7fc1bc00cd10 comp rx=0 tx=0).stop 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 shutdown_connections 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1b00a2fb0 0x7fc1b00a5470 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1c81013e0 0x7fc1c8198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1c81026a0 0x7fc1c819f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 --2- 172.21.15.87:0/4010069663 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1c8101d80 0x7fc1c8198990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:16.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.712+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 >> 172.21.15.87:0/4010069663 conn(0x7fc1c80fbe30 msgr2=0x7fc1c80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:16.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.713+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 shutdown_connections 2024-03-21T08:54:16.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:16.713+0000 7fc1cd3cb700 1 -- 172.21.15.87:0/4010069663 wait complete. 2024-03-21T08:54:16.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:16 smithi105 ceph-mon[35993]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:54:16.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:16 smithi105 ceph-mon[35993]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T08:54:16.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:16.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:16 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:16 smithi100 ceph-mon[36306]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:54:17.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:16 smithi100 ceph-mon[36306]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T08:54:17.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:16 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.071 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:16 smithi087 ceph-mon[32385]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-03-21T08:54:17.071 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:16 smithi087 ceph-mon[32385]: osdmap e80: 12 total, 11 up, 12 in 2024-03-21T08:54:17.071 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.071 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:16 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.550 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1711011256,"num_in_osds":12,"osd_in_since":1711011236,"num_remapped_pgs":0} 2024-03-21T08:54:17.551 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T08:54:17.803 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: purged_snaps scrub starts 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: purged_snaps scrub ok 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] boot 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4010069663' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.820 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.820 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.820 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:17.820 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:17.820 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:17 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: purged_snaps scrub starts 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: purged_snaps scrub ok 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] boot 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4010069663' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:17.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:17 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:18.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: purged_snaps scrub starts 2024-03-21T08:54:18.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: purged_snaps scrub ok 2024-03-21T08:54:18.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: osd.11 [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] boot 2024-03-21T08:54:18.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: osdmap e81: 12 total, 12 up, 12 in 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4010069663' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:18.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:17 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:18.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:18 smithi105 ceph-mon[35993]: Detected new or changed devices on smithi105 2024-03-21T08:54:18.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:18 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-21T08:54:18.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:18 smithi105 ceph-mon[35993]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T08:54:19.002 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:18 smithi087 ceph-mon[32385]: Detected new or changed devices on smithi105 2024-03-21T08:54:19.002 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:18 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-21T08:54:19.002 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:18 smithi087 ceph-mon[32385]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T08:54:19.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:18 smithi100 ceph-mon[36306]: Detected new or changed devices on smithi105 2024-03-21T08:54:19.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:18 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi105 to 4270M 2024-03-21T08:54:19.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:18 smithi100 ceph-mon[36306]: osdmap e82: 12 total, 12 up, 12 in 2024-03-21T08:54:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 -- 172.21.15.87:0/380259935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c1021e0 msgr2=0x7f8e7c102660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/380259935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c1021e0 0x7f8e7c102660 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f8e640099c0 tx=0x7f8e64009cd0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.544 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 -- 172.21.15.87:0/380259935 shutdown_connections 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/380259935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c10efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/380259935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e7c1081c0 0x7f8e7c1085a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/380259935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c1021e0 0x7f8e7c102660 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 -- 172.21.15.87:0/380259935 >> 172.21.15.87:0/380259935 conn(0x7f8e7c0792c0 msgr2=0x7f8e7c0796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 -- 172.21.15.87:0/380259935 shutdown_connections 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.543+0000 7f8e81ab3700 1 -- 172.21.15.87:0/380259935 wait complete. 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 Processor -- start 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 -- start start 2024-03-21T08:54:19.545 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c1021e0 0x7f8e7c19e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c19e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e7c1081c0 0x7f8e7c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c112460 con 0x7f8e7c102ba0 2024-03-21T08:54:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1122e0 con 0x7f8e7c1021e0 2024-03-21T08:54:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e81ab3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e7c1125e0 con 0x7f8e7c1081c0 2024-03-21T08:54:19.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e72ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c19e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e72ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c19e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34280/0 (socket says 172.21.15.87:34280) 2024-03-21T08:54:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e72ffd700 1 -- 172.21.15.87:0/4183152270 learned_addr learned my addr 172.21.15.87:0/4183152270 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.544+0000 7f8e7bfff700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e7c1081c0 0x7f8e7c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e7b7fe700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c1021e0 0x7f8e7c19e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:19.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e72ffd700 1 -- 172.21.15.87:0/4183152270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e7c1081c0 msgr2=0x7f8e7c198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e72ffd700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e7c1081c0 0x7f8e7c198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e72ffd700 1 -- 172.21.15.87:0/4183152270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c1021e0 msgr2=0x7f8e7c19e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e72ffd700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c1021e0 0x7f8e7c19e460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e72ffd700 1 -- 172.21.15.87:0/4183152270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e7c198d80 con 0x7f8e7c102ba0 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e72ffd700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c19e9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e64006010 tx=0x7f8e64009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64027070 con 0x7f8e7c102ba0 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e64003780 con 0x7f8e7c102ba0 2024-03-21T08:54:19.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e7c199010 con 0x7f8e7c102ba0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e64022da0 con 0x7f8e7c102ba0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.545+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e7c1a5330 con 0x7f8e7c102ba0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.546+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e7c103870 con 0x7f8e7c102ba0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.547+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8e6402a070 con 0x7f8e7c102ba0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.548+0000 7f8e797fa700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e680a2fb0 0x7f8e680a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.548+0000 7f8e7b7fe700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e680a2fb0 0x7f8e680a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.548+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8e640cfd30 con 0x7f8e7c102ba0 2024-03-21T08:54:19.549 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.548+0000 7f8e7b7fe700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e680a2fb0 0x7f8e680a5470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005950 tx=0x7f8e6c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:19.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.552+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e6405e740 con 0x7f8e7c102ba0 2024-03-21T08:54:19.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.749+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8e7c1a5500 con 0x7f8e7c102ba0 2024-03-21T08:54:19.746 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.750+0000 7f8e797fa700 1 -- 172.21.15.87:0/4183152270 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f8e64013070 con 0x7f8e7c102ba0 2024-03-21T08:54:19.747 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:19.747 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":82,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:54:17.710738+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:54:19.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e680a2fb0 msgr2=0x7f8e680a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:19.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e680a2fb0 0x7f8e680a5470 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c005950 tx=0x7f8e6c0058e0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 msgr2=0x7f8e7c19e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:19.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c19e9a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8e64006010 tx=0x7f8e64009f80 comp rx=0 tx=0).stop 2024-03-21T08:54:19.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 shutdown_connections 2024-03-21T08:54:19.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e680a2fb0 0x7f8e680a5470 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e7c1021e0 0x7f8e7c19e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e7c1081c0 0x7f8e7c198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 --2- 172.21.15.87:0/4183152270 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e7c102ba0 0x7f8e7c19e9a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:19.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 >> 172.21.15.87:0/4183152270 conn(0x7f8e7c0792c0 msgr2=0x7f8e7c100c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:19.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 shutdown_connections 2024-03-21T08:54:19.750 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:19.753+0000 7f8e81ab3700 1 -- 172.21.15.87:0/4183152270 wait complete. 2024-03-21T08:54:19.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:19 smithi105 ceph-mon[35993]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:20.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:19 smithi100 ceph-mon[36306]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:20.091 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:19 smithi087 ceph-mon[32385]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:20.396 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:54:20.397 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-21T08:54:20.632 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:54:20.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:20 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4183152270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:54:20.996 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:20 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4183152270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:54:21.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:20 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4183152270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:54:21.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:21 smithi105 ceph-mon[35993]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:21.985 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:21 smithi087 ceph-mon[32385]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:22.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:21 smithi100 ceph-mon[36306]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:22.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- 172.21.15.87:0/3145291426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4107520 msgr2=0x7fa7e410f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/3145291426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4107520 0x7fa7e410f0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fa7dc00b0d0 tx=0x7fa7dc00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- 172.21.15.87:0/3145291426 shutdown_connections 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/3145291426 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4107520 0x7fa7e410f0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/3145291426 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e4102470 0x7fa7e4102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/3145291426 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7e4102e20 0x7fa7e4106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- 172.21.15.87:0/3145291426 >> 172.21.15.87:0/3145291426 conn(0x7fa7e40792b0 msgr2=0x7fa7e40796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- 172.21.15.87:0/3145291426 shutdown_connections 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- 172.21.15.87:0/3145291426 wait complete. 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 Processor -- start 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- start start 2024-03-21T08:54:22.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7e4102470 0x7fa7e4074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 0x7fa7e40749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e4107520 0x7fa7e4074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e41124e0 con 0x7fa7e4102e20 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4112360 con 0x7fa7e4102470 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.320+0000 7fa7eac6d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e4112660 con 0x7fa7e4107520 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 0x7fa7e40749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 0x7fa7e40749d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34298/0 (socket says 172.21.15.87:34298) 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 -- 172.21.15.87:0/12725302 learned_addr learned my addr 172.21.15.87:0/12725302 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e920a700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e4107520 0x7fa7e4074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e8a09700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7e4102470 0x7fa7e4074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 -- 172.21.15.87:0/12725302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e4107520 msgr2=0x7fa7e4074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e4107520 0x7fa7e4074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 -- 172.21.15.87:0/12725302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7e4102470 msgr2=0x7fa7e4074490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7e4102470 0x7fa7e4074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 -- 172.21.15.87:0/12725302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e41a6e20 con 0x7fa7e4102e20 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e3fff700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 0x7fa7e40749d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000d950 tx=0x7fa7d000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0017070 con 0x7fa7e4102e20 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa7d0009e10 con 0x7fa7e4102e20 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e41a7110 con 0x7fa7e4102e20 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000f3c0 con 0x7fa7e4102e20 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.321+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e41a76a0 con 0x7fa7e4102e20 2024-03-21T08:54:22.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.322+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e410a6f0 con 0x7fa7e4102e20 2024-03-21T08:54:22.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.323+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa7d001a070 con 0x7fa7e4102e20 2024-03-21T08:54:22.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.324+0000 7fa7e1ffb700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa7d40a3030 0x7fa7d40a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:22.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.324+0000 7fa7e8a09700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa7d40a3030 0x7fa7d40a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:22.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.324+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa7d00ce6a0 con 0x7fa7e4102e20 2024-03-21T08:54:22.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.324+0000 7fa7e8a09700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa7d40a3030 0x7fa7d40a54f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8005950 tx=0x7fa7d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:22.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.328+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa7d009b9f0 con 0x7fa7e4102e20 2024-03-21T08:54:22.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.521+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7e402da30 con 0x7fa7e4102e20 2024-03-21T08:54:22.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.522+0000 7fa7e1ffb700 1 -- 172.21.15.87:0/12725302 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7d009b2f0 con 0x7fa7e4102e20 2024-03-21T08:54:22.518 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:54:22.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa7d40a3030 msgr2=0x7fa7d40a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:22.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa7d40a3030 0x7fa7d40a54f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8005950 tx=0x7fa7d80058e0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 msgr2=0x7fa7e40749d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 0x7fa7e40749d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000d950 tx=0x7fa7d000dc60 comp rx=0 tx=0).stop 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 shutdown_connections 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa7d40a3030 0x7fa7d40a54f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa7e4102470 0x7fa7e4074490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa7e4107520 0x7fa7e4074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 --2- 172.21.15.87:0/12725302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa7e4102e20 0x7fa7e40749d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 >> 172.21.15.87:0/12725302 conn(0x7fa7e40792b0 msgr2=0x7fa7e41002c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 shutdown_connections 2024-03-21T08:54:22.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:22.524+0000 7fa7eac6d700 1 -- 172.21.15.87:0/12725302 wait complete. 2024-03-21T08:54:22.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:22 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/12725302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:54:23.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:22 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/12725302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:54:23.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:22 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/12725302' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.a on smithi087 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.d on smithi087 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.g on smithi087 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.j on smithi087 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.b on smithi100 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.e on smithi100 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.h on smithi100 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.k on smithi100 2024-03-21T08:54:23.305 INFO:tasks.cephadm:Adding mds.c on smithi105 2024-03-21T08:54:23.306 INFO:tasks.cephadm:Adding mds.f on smithi105 2024-03-21T08:54:23.306 INFO:tasks.cephadm:Adding mds.i on smithi105 2024-03-21T08:54:23.306 INFO:tasks.cephadm:Adding mds.l on smithi105 2024-03-21T08:54:23.306 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph orch apply mds all '12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l' 2024-03-21T08:54:23.553 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T08:54:23.932 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:23 smithi105 ceph-mon[35993]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:24.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:23 smithi100 ceph-mon[36306]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:24.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:23 smithi087 ceph-mon[32385]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/1341144187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b88103d60 msgr2=0x7f0b8810a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/1341144187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b88103d60 0x7f0b8810a610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0b8000b0a0 tx=0x7f0b8000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/1341144187 shutdown_connections 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/1341144187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b881021c0 0x7f0b881025e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/1341144187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b88103d60 0x7f0b8810a610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/1341144187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b881033c0 0x7f0b88103820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.350 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/1341144187 >> 172.21.15.105:0/1341144187 conn(0x7f0b880fd7a0 msgr2=0x7f0b880ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/1341144187 shutdown_connections 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.352+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/1341144187 wait complete. 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 Processor -- start 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 -- start start 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b881021c0 0x7f0b88198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 0x7f0b88198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b88103d60 0x7f0b8819f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8810ddc0 con 0x7f0b88103d60 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8810dc40 con 0x7f0b881021c0 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8f9a9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8810df40 con 0x7f0b881033c0 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8cf44700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 0x7f0b88198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8cf44700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 0x7f0b88198880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.105:42798/0 (socket says 172.21.15.105:42798) 2024-03-21T08:54:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8cf44700 1 -- 172.21.15.105:0/464870834 learned_addr learned my addr 172.21.15.105:0/464870834 (peer_addr_for_me v2:172.21.15.105:0/0) 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8d745700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b881021c0 0x7f0b88198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.353+0000 7f0b8df46700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b88103d60 0x7f0b8819f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8cf44700 1 -- 172.21.15.105:0/464870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b881021c0 msgr2=0x7f0b88198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8cf44700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b881021c0 0x7f0b88198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8cf44700 1 -- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b88103d60 msgr2=0x7f0b8819f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8cf44700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b88103d60 0x7f0b8819f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8cf44700 1 -- 172.21.15.105:0/464870834 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8819ffa0 con 0x7f0b881033c0 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8df46700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b88103d60 0x7f0b8819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8cf44700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 0x7f0b88198880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400e9d0 tx=0x7f0b8400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400cbd0 con 0x7f0b881033c0 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b8400cd70 con 0x7f0b881033c0 2024-03-21T08:54:25.352 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b881a0290 con 0x7f0b881033c0 2024-03-21T08:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84018660 con 0x7f0b881033c0 2024-03-21T08:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.354+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b881a07d0 con 0x7f0b881033c0 2024-03-21T08:54:25.353 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.355+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b88068e50 con 0x7f0b881033c0 2024-03-21T08:54:25.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.356+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0b84018840 con 0x7f0b881033c0 2024-03-21T08:54:25.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.357+0000 7f0b7e7fc700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b740a2fb0 0x7f0b740a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:25.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.357+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0b840cf230 con 0x7f0b881033c0 2024-03-21T08:54:25.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.357+0000 7f0b8d745700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b740a2fb0 0x7f0b740a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:25.354 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.358+0000 7f0b8d745700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b740a2fb0 0x7f0b740a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005950 tx=0x7f0b7800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:25.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.361+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b8409c500 con 0x7f0b881033c0 2024-03-21T08:54:25.589 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.593+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b88062890 con 0x7f0b740a2fb0 2024-03-21T08:54:25.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.603+0000 7f0b7e7fc700 1 -- 172.21.15.105:0/464870834 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f0b88062890 con 0x7f0b740a2fb0 2024-03-21T08:54:25.599 INFO:teuthology.orchestra.run.smithi105.stdout:Scheduled mds.all update... 2024-03-21T08:54:25.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b740a2fb0 msgr2=0x7f0b740a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:25.601 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b740a2fb0 0x7f0b740a5470 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f0b78005950 tx=0x7f0b7800a300 comp rx=0 tx=0).stop 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 msgr2=0x7f0b88198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 0x7f0b88198880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b8400e9d0 tx=0x7f0b8400ece0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 shutdown_connections 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b740a2fb0 0x7f0b740a5470 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b881021c0 0x7f0b88198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b88103d60 0x7f0b8819f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 --2- 172.21.15.105:0/464870834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b881033c0 0x7f0b88198880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.605+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 >> 172.21.15.105:0/464870834 conn(0x7f0b880fd7a0 msgr2=0x7f0b881066c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.606+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 shutdown_connections 2024-03-21T08:54:25.602 INFO:teuthology.orchestra.run.smithi105.stderr:2024-03-21T08:54:25.606+0000 7f0b8f9a9700 1 -- 172.21.15.105:0/464870834 wait complete. 2024-03-21T08:54:26.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:25 smithi087 ceph-mon[32385]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:26.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:26.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:26.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:25 smithi105 ceph-mon[35993]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:26.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:26.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:25 smithi100 ceph-mon[36306]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:26.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:26.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:54:26.323 DEBUG:teuthology.orchestra.run.smithi087:mds.a> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.a.service 2024-03-21T08:54:26.326 DEBUG:teuthology.orchestra.run.smithi087:mds.d> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.d.service 2024-03-21T08:54:26.369 DEBUG:teuthology.orchestra.run.smithi087:mds.g> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.g.service 2024-03-21T08:54:26.412 DEBUG:teuthology.orchestra.run.smithi087:mds.j> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.j.service 2024-03-21T08:54:26.457 DEBUG:teuthology.orchestra.run.smithi100:mds.b> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.b.service 2024-03-21T08:54:26.460 DEBUG:teuthology.orchestra.run.smithi100:mds.e> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.e.service 2024-03-21T08:54:26.502 DEBUG:teuthology.orchestra.run.smithi100:mds.h> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.h.service 2024-03-21T08:54:26.545 DEBUG:teuthology.orchestra.run.smithi100:mds.k> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.k.service 2024-03-21T08:54:26.588 DEBUG:teuthology.orchestra.run.smithi105:mds.c> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.c.service 2024-03-21T08:54:26.591 DEBUG:teuthology.orchestra.run.smithi105:mds.f> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.f.service 2024-03-21T08:54:26.634 DEBUG:teuthology.orchestra.run.smithi105:mds.i> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.i.service 2024-03-21T08:54:26.677 DEBUG:teuthology.orchestra.run.smithi105:mds.l> sudo journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.l.service 2024-03-21T08:54:26.720 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-03-21T08:54:26.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:54:26.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- 172.21.15.87:0/597649747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c104a70 msgr2=0x7f974c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:26.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- 172.21.15.87:0/597649747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c104a70 0x7f974c106e60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f973c009900 tx=0x7f973c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- 172.21.15.87:0/597649747 shutdown_connections 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- 172.21.15.87:0/597649747 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 0x7f974c1097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- 172.21.15.87:0/597649747 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c104a70 0x7f974c106e60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- 172.21.15.87:0/597649747 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c102140 0x7f974c104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- 172.21.15.87:0/597649747 >> 172.21.15.87:0/597649747 conn(0x7f974c0fbb30 msgr2=0x7f974c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- 172.21.15.87:0/597649747 shutdown_connections 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- 172.21.15.87:0/597649747 wait complete. 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 Processor -- start 2024-03-21T08:54:26.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- start start 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 0x7f974c101ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c104a70 0x7f974c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 0x7f974c100630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c10d3a0 con 0x7f974c102140 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c10d220 con 0x7f974c1073a0 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.835+0000 7f975444b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974c10d520 con 0x7f974c104a70 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 0x7f974c101ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 0x7f974c101ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34324/0 (socket says 172.21.15.87:34324) 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 -- 172.21.15.87:0/232106824 learned_addr learned my addr 172.21.15.87:0/232106824 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97519e6640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c104a70 0x7f974c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97529e8640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 0x7f974c100630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 -- 172.21.15.87:0/232106824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c104a70 msgr2=0x7f974c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c104a70 0x7f974c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:26.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 -- 172.21.15.87:0/232106824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 msgr2=0x7f974c100630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 0x7f974c100630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 -- 172.21.15.87:0/232106824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974c100ec0 con 0x7f974c102140 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97529e8640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 0x7f974c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97521e7640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 0x7f974c101ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f974800e9f0 tx=0x7f974800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f974800cd80 con 0x7f974c102140 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9748005500 con 0x7f974c102140 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974c1a1b10 con 0x7f974c102140 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9748010690 con 0x7f974c102140 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.836+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f974c1a2030 con 0x7f974c102140 2024-03-21T08:54:26.836 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.837+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974c068490 con 0x7f974c102140 2024-03-21T08:54:26.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.838+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f97480046c0 con 0x7f974c102140 2024-03-21T08:54:26.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.839+0000 7f97437fe640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97380a2fa0 0x7f97380a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:26.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.839+0000 7f97519e6640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97380a2fa0 0x7f97380a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:26.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.839+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f97480cf550 con 0x7f974c102140 2024-03-21T08:54:26.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.839+0000 7f97519e6640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97380a2fa0 0x7f97380a5460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f973c02c170 tx=0x7f973c02df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:26.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:26.843+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f9748053740 con 0x7f974c102140 2024-03-21T08:54:27.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.019+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f974c02d070 con 0x7f974c102140 2024-03-21T08:54:27.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.020+0000 7f97437fe640 1 -- 172.21.15.87:0/232106824 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18466 (secure 0 0 0) 0x7f974809c820 con 0x7f974c102140 2024-03-21T08:54:27.017 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:27.017 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":82,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:54:17.710738+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97380a2fa0 msgr2=0x7f97380a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97380a2fa0 0x7f97380a5460 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f973c02c170 tx=0x7f973c02df90 comp rx=0 tx=0).stop 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 msgr2=0x7f974c101ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 0x7f974c101ab0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f974800e9f0 tx=0x7f974800edf0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 shutdown_connections 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97380a2fa0 0x7f97380a5460 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f974c1073a0 0x7f974c100630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f974c104a70 0x7f974c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 --2- 172.21.15.87:0/232106824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f974c102140 0x7f974c101ab0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 >> 172.21.15.87:0/232106824 conn(0x7f974c0fbb30 msgr2=0x7f974c102b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:27.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 shutdown_connections 2024-03-21T08:54:27.020 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.022+0000 7f975444b640 1 -- 172.21.15.87:0/232106824 wait complete. 2024-03-21T08:54:27.028 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:54:27.028 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: Saving service mds.all spec with placement smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l;count:12 2024-03-21T08:54:27.029 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:27.029 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:27.029 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:27.029 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:27.029 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:27.029 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:26 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:27.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:54:27.030 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 -- 172.21.15.87:0/1207350941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50102150 msgr2=0x7f7f501089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/1207350941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50102150 0x7f7f501089e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7f48009900 tx=0x7f7f4802bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 -- 172.21.15.87:0/1207350941 shutdown_connections 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/1207350941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50102150 0x7f7f501089e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/1207350941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f50101790 0x7f7f50101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/1207350941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f501005a0 0x7f7f501009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 -- 172.21.15.87:0/1207350941 >> 172.21.15.87:0/1207350941 conn(0x7f7f500fbb30 msgr2=0x7f7f500fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:27.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 -- 172.21.15.87:0/1207350941 shutdown_connections 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.139+0000 7f7f57e0b640 1 -- 172.21.15.87:0/1207350941 wait complete. 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 Processor -- start 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 -- start start 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f501005a0 0x7f7f50104530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 0x7f7f50102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f50102150 0x7f7f501030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5010c230 con 0x7f7f50101790 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5010c0b0 con 0x7f7f501005a0 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f57e0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f5010c3b0 con 0x7f7f50102150 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 0x7f7f50102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 0x7f7f50102b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34352/0 (socket says 172.21.15.87:34352) 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 -- 172.21.15.87:0/700576538 learned_addr learned my addr 172.21.15.87:0/700576538 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:27.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f55ba7640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f501005a0 0x7f7f50104530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f563a8640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f50102150 0x7f7f501030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 -- 172.21.15.87:0/700576538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f50102150 msgr2=0x7f7f501030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f50102150 0x7f7f501030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 -- 172.21.15.87:0/700576538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f501005a0 msgr2=0x7f7f50104530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f501005a0 0x7f7f50104530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.140+0000 7f7f553a6640 1 -- 172.21.15.87:0/700576538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f50103940 con 0x7f7f50101790 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.141+0000 7f7f553a6640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 0x7f7f50102b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400d950 tx=0x7f7f4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.141+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f44017030 con 0x7f7f50101790 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.141+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f501a1aa0 con 0x7f7f50101790 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.141+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f44004020 con 0x7f7f50101790 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.141+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f440047a0 con 0x7f7f50101790 2024-03-21T08:54:27.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.141+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f501a2040 con 0x7f7f50101790 2024-03-21T08:54:27.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.142+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f501a2270 con 0x7f7f50101790 2024-03-21T08:54:27.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.143+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f440041d0 con 0x7f7f50101790 2024-03-21T08:54:27.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.143+0000 7f7f4effd640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.144+0000 7f7f55ba7640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.144+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7f440ce970 con 0x7f7f50101790 2024-03-21T08:54:27.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.144+0000 7f7f55ba7640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f501015f0 tx=0x7f7f4000a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:27.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.148+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f4409bc40 con 0x7f7f50101790 2024-03-21T08:54:27.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:54:27.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: Saving service mds.all spec with placement smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l;count:12 2024-03-21T08:54:27.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:27.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:26 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: Saving service mds.all spec with placement smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l;count:12 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:27.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:27.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:26 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:27.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.321+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7f501a2440 con 0x7f7f50101790 2024-03-21T08:54:27.318 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.322+0000 7f7f4effd640 1 -- 172.21.15.87:0/700576538 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7f4409b540 con 0x7f7f50101790 2024-03-21T08:54:27.319 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 msgr2=0x7f7f380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7f501015f0 tx=0x7f7f4000a590 comp rx=0 tx=0).stop 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 msgr2=0x7f7f50102b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 0x7f7f50102b70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7f4400d950 tx=0x7f7f4400de10 comp rx=0 tx=0).stop 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 shutdown_connections 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f50102150 0x7f7f501030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f50101790 0x7f7f50102b70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 --2- 172.21.15.87:0/700576538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f501005a0 0x7f7f50104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 >> 172.21.15.87:0/700576538 conn(0x7f7f500fbb30 msgr2=0x7f7f500fd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:27.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 shutdown_connections 2024-03-21T08:54:27.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.324+0000 7f7f57e0b640 1 -- 172.21.15.87:0/700576538 wait complete. 2024-03-21T08:54:27.331 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- 172.21.15.87:0/4157999748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 msgr2=0x7fdfa8100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- 172.21.15.87:0/4157999748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa8100170 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fdf9800aa10 tx=0x7fdf9802cf70 comp rx=0 tx=0).stop 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- 172.21.15.87:0/4157999748 shutdown_connections 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- 172.21.15.87:0/4157999748 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfa8101900 0x7fdfa8108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- 172.21.15.87:0/4157999748 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 0x7fdfa81013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- 172.21.15.87:0/4157999748 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa8100170 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- 172.21.15.87:0/4157999748 >> 172.21.15.87:0/4157999748 conn(0x7fdfa80fb2c0 msgr2=0x7fdfa80fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- 172.21.15.87:0/4157999748 shutdown_connections 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- 172.21.15.87:0/4157999748 wait complete. 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 Processor -- start 2024-03-21T08:54:27.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- start start 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa81957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 0x7fdfa8195d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfa8101900 0x7fdfa819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa810b160 con 0x7fdfa80ffd50 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa810afe0 con 0x7fdfa8100f40 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfaf861640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa810b2e0 con 0x7fdfa8101900 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.438+0000 7fdfad5fd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa81957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa81957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34376/0 (socket says 172.21.15.87:34376) 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 -- 172.21.15.87:0/396687080 learned_addr learned my addr 172.21.15.87:0/396687080 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfaddfe640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfa8101900 0x7fdfa819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfacdfc640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 0x7fdfa8195d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 -- 172.21.15.87:0/396687080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfa8101900 msgr2=0x7fdfa819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfa8101900 0x7fdfa819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 -- 172.21.15.87:0/396687080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 msgr2=0x7fdfa8195d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 0x7fdfa8195d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 -- 172.21.15.87:0/396687080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa819d410 con 0x7fdfa80ffd50 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfacdfc640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 0x7fdfa8195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfad5fd640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa81957f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdf98000c00 tx=0x7fdf98005230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98044020 con 0x7fdfa80ffd50 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fdf980048d0 con 0x7fdfa80ffd50 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa819d6a0 con 0x7fdfa80ffd50 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf98003850 con 0x7fdfa80ffd50 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.439+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa819dbf0 con 0x7fdfa80ffd50 2024-03-21T08:54:27.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.440+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa8107070 con 0x7fdfa80ffd50 2024-03-21T08:54:27.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.441+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fdf98047020 con 0x7fdfa80ffd50 2024-03-21T08:54:27.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.442+0000 7fdfa67fc640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fdf900a2f20 0x7fdf900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:27.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.442+0000 7fdfacdfc640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fdf900a2f20 0x7fdf900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:27.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.442+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdf980f2870 con 0x7fdfa80ffd50 2024-03-21T08:54:27.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.442+0000 7fdfacdfc640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fdf900a2f20 0x7fdf900a53e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8196d00 tx=0x7fdf9c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:27.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.446+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fdf9804e020 con 0x7fdfa80ffd50 2024-03-21T08:54:27.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.660+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfa802d070 con 0x7fdfa80ffd50 2024-03-21T08:54:27.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.661+0000 7fdfa67fc640 1 -- 172.21.15.87:0/396687080 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+535 (secure 0 0 0) 0x7fdf980df020 con 0x7fdfa80ffd50 2024-03-21T08:54:27.657 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:27.657 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-03-21T08:54:27.658 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 1 2024-03-21T08:54:27.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fdf900a2f20 msgr2=0x7fdf900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fdf900a2f20 0x7fdf900a53e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fdfa8196d00 tx=0x7fdf9c009500 comp rx=0 tx=0).stop 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 msgr2=0x7fdfa81957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa81957f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdf98000c00 tx=0x7fdf98005230 comp rx=0 tx=0).stop 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 shutdown_connections 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fdf900a2f20 0x7fdf900a53e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fdfa8101900 0x7fdfa819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fdfa8100f40 0x7fdfa8195d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 --2- 172.21.15.87:0/396687080 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fdfa80ffd50 0x7fdfa81957f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 >> 172.21.15.87:0/396687080 conn(0x7fdfa80fb2c0 msgr2=0x7fdfa8104240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 shutdown_connections 2024-03-21T08:54:27.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:27.663+0000 7fdfaf861640 1 -- 172.21.15.87:0/396687080 wait complete. 2024-03-21T08:54:28.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:27 smithi087 ceph-mon[32385]: Deploying daemon mds.l on smithi105 2024-03-21T08:54:28.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:27 smithi087 ceph-mon[32385]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:28.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:27 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/232106824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:54:28.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:27 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/700576538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:54:28.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:27 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/396687080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:28.186 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:27 smithi105 ceph-mon[35993]: Deploying daemon mds.l on smithi105 2024-03-21T08:54:28.186 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:27 smithi105 ceph-mon[35993]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:28.187 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:27 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/232106824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:54:28.187 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:27 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/700576538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:54:28.187 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:27 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/396687080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:27 smithi100 ceph-mon[36306]: Deploying daemon mds.l on smithi105 2024-03-21T08:54:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:27 smithi100 ceph-mon[36306]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:27 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/232106824' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:54:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:27 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/700576538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:54:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:27 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/396687080' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:29.483 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 08:54:29 smithi105 systemd[1]: Started Ceph mds.l for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:29.671 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 -- 172.21.15.87:0/4101922406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a8104a70 msgr2=0x7f82a8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/4101922406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a8104a70 0x7f82a8106e60 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f829c009900 tx=0x7f829c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 -- 172.21.15.87:0/4101922406 shutdown_connections 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/4101922406 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a81073a0 0x7f82a81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/4101922406 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a8104a70 0x7f82a8106e60 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/4101922406 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 0x7f82a8104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 -- 172.21.15.87:0/4101922406 >> 172.21.15.87:0/4101922406 conn(0x7f82a80fbb50 msgr2=0x7f82a80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 -- 172.21.15.87:0/4101922406 shutdown_connections 2024-03-21T08:54:29.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.788+0000 7f82ad9e6640 1 -- 172.21.15.87:0/4101922406 wait complete. 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 Processor -- start 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 -- start start 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 0x7f82a81979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a8104a70 0x7f82a8197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 0x7f82a819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a810d340 con 0x7f82a81073a0 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a810d1c0 con 0x7f82a8102140 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82ad9e6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82a810d4c0 con 0x7f82a8104a70 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 0x7f82a819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 0x7f82a819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54892/0 (socket says 172.21.15.87:54892) 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 -- 172.21.15.87:0/2442530799 learned_addr learned my addr 172.21.15.87:0/2442530799 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a6ffd640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 0x7f82a81979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:29.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a67fc640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a8104a70 0x7f82a8197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 -- 172.21.15.87:0/2442530799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a8104a70 msgr2=0x7f82a8197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a8104a70 0x7f82a8197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 -- 172.21.15.87:0/2442530799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 msgr2=0x7f82a81979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 0x7f82a81979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a77fe640 1 -- 172.21.15.87:0/2442530799 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a819f610 con 0x7f82a81073a0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.789+0000 7f82a6ffd640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 0x7f82a81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.790+0000 7f82a77fe640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 0x7f82a819efa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f82a000eaa0 tx=0x7f82a000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.790+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a000cdb0 con 0x7f82a81073a0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.790+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f82a0005020 con 0x7f82a81073a0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.790+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82a819f900 con 0x7f82a81073a0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.790+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a00116b0 con 0x7f82a81073a0 2024-03-21T08:54:29.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.790+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82a819fe20 con 0x7f82a81073a0 2024-03-21T08:54:29.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.791+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82a80fd640 con 0x7f82a81073a0 2024-03-21T08:54:29.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.792+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f82a0005560 con 0x7f82a81073a0 2024-03-21T08:54:29.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.792+0000 7f82ac9e4640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f82900a2f20 0x7f82900a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:29.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.793+0000 7f82a6ffd640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f82900a2f20 0x7f82900a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:29.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.793+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f82a00cf0f0 con 0x7f82a81073a0 2024-03-21T08:54:29.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.793+0000 7f82a6ffd640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f82900a2f20 0x7f82900a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8298009fc0 tx=0x7f8298005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:29.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:29.797+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f82a0016030 con 0x7f82a81073a0 2024-03-21T08:54:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.012+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82a80622a0 con 0x7f82a81073a0 2024-03-21T08:54:30.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.013+0000 7f82ac9e4640 1 -- 172.21.15.87:0/2442530799 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7f82a009bd70 con 0x7f82a81073a0 2024-03-21T08:54:30.009 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:30.009 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:54:30.010 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 2 2024-03-21T08:54:30.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f82900a2f20 msgr2=0x7f82900a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f82900a2f20 0x7f82900a53e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f8298009fc0 tx=0x7f8298005f90 comp rx=0 tx=0).stop 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 msgr2=0x7f82a819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 0x7f82a819efa0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f82a000eaa0 tx=0x7f82a000ef60 comp rx=0 tx=0).stop 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 shutdown_connections 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f82900a2f20 0x7f82900a53e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f82a81073a0 0x7f82a819efa0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f82a8104a70 0x7f82a8197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 --2- 172.21.15.87:0/2442530799 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f82a8102140 0x7f82a81979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 >> 172.21.15.87:0/2442530799 conn(0x7f82a80fbb50 msgr2=0x7f82a81029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 shutdown_connections 2024-03-21T08:54:30.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:30.015+0000 7f82ad9e6640 1 -- 172.21.15.87:0/2442530799 wait complete. 2024-03-21T08:54:30.021 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:30.021 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] up:boot 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: fsmap 1 up:standby 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:30.022 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:29 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:30.155 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:30.155 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] up:boot 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: fsmap 1 up:standby 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:30.156 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:29 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] up:boot 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: fsmap 1 up:standby 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:30.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:30.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:29 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:31.126 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:30 smithi100 ceph-mon[36306]: Deploying daemon mds.k on smithi100 2024-03-21T08:54:31.127 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:30 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2442530799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:31.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:30 smithi105 ceph-mon[35993]: Deploying daemon mds.k on smithi100 2024-03-21T08:54:31.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:30 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2442530799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:31.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:30 smithi087 ceph-mon[32385]: Deploying daemon mds.k on smithi100 2024-03-21T08:54:31.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:30 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2442530799' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:31.911 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:31 smithi100 podman[65239]: 2024-03-21 08:54:31.774750371 +0000 UTC m=+0.136240443 container create 69d3153453cac6a2c8a0eb7fa5cc380f01d21d9a535f1b204682f7384b3f7e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:54:32.024 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 -- 172.21.15.87:0/968062160 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c0ffd50 msgr2=0x7fad8c100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 --2- 172.21.15.87:0/968062160 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c0ffd50 0x7fad8c100170 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fad80009900 tx=0x7fad8002bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 -- 172.21.15.87:0/968062160 shutdown_connections 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 --2- 172.21.15.87:0/968062160 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c101900 0x7fad8c108190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 --2- 172.21.15.87:0/968062160 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad8c100f40 0x7fad8c1013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 --2- 172.21.15.87:0/968062160 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c0ffd50 0x7fad8c100170 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 -- 172.21.15.87:0/968062160 >> 172.21.15.87:0/968062160 conn(0x7fad8c0fb2c0 msgr2=0x7fad8c0fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:32.138 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 -- 172.21.15.87:0/968062160 shutdown_connections 2024-03-21T08:54:32.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.141+0000 7fad9084b640 1 -- 172.21.15.87:0/968062160 wait complete. 2024-03-21T08:54:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 Processor -- start 2024-03-21T08:54:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 -- start start 2024-03-21T08:54:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c0ffd50 0x7fad8c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 0x7fad8c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad8c101900 0x7fad8c19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:32.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c10c240 con 0x7fad8c100f40 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c10c0c0 con 0x7fad8c0ffd50 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad9084b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad8c10c3c0 con 0x7fad8c101900 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad89d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 0x7fad8c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad89d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 0x7fad8c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54908/0 (socket says 172.21.15.87:54908) 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad89d9b640 1 -- 172.21.15.87:0/1574417541 learned_addr learned my addr 172.21.15.87:0/1574417541 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad8a59c640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c0ffd50 0x7fad8c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.142+0000 7fad8ad9d640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad8c101900 0x7fad8c19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad89d9b640 1 -- 172.21.15.87:0/1574417541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad8c101900 msgr2=0x7fad8c19eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad89d9b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad8c101900 0x7fad8c19eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad89d9b640 1 -- 172.21.15.87:0/1574417541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c0ffd50 msgr2=0x7fad8c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad89d9b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c0ffd50 0x7fad8c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad89d9b640 1 -- 172.21.15.87:0/1574417541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad8c19f660 con 0x7fad8c100f40 2024-03-21T08:54:32.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad8a59c640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c0ffd50 0x7fad8c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad89d9b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 0x7fad8c197f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d770 tx=0x7fad7c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c017070 con 0x7fad8c100f40 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad7c005500 con 0x7fad8c100f40 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad8c19f950 con 0x7fad8c100f40 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7c004e00 con 0x7fad8c100f40 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.143+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad8c19fe50 con 0x7fad8c100f40 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.144+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad8c1a0080 con 0x7fad8c100f40 2024-03-21T08:54:32.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.145+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fad7c004fa0 con 0x7fad8c100f40 2024-03-21T08:54:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.145+0000 7fad7b7fe640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad740a2f20 0x7fad740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.146+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fad7c0cea20 con 0x7fad8c100f40 2024-03-21T08:54:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.146+0000 7fad8a59c640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad740a2f20 0x7fad740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:32.144 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.146+0000 7fad8a59c640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad740a2f20 0x7fad740a53e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad8002c7b0 tx=0x7fad80005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:32.146 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.150+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fad7c09bcf0 con 0x7fad8c100f40 2024-03-21T08:54:32.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:31 smithi105 ceph-mon[35993]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:32.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:31 smithi100 ceph-mon[36306]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:32.267 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:32 smithi100 podman[65239]: 2024-03-21 08:54:32.027759764 +0000 UTC m=+0.389249854 container init 69d3153453cac6a2c8a0eb7fa5cc380f01d21d9a535f1b204682f7384b3f7e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T08:54:32.267 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:32 smithi100 podman[65239]: 2024-03-21 08:54:32.030811485 +0000 UTC m=+0.392301564 container start 69d3153453cac6a2c8a0eb7fa5cc380f01d21d9a535f1b204682f7384b3f7e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T08:54:32.267 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:32 smithi100 ceph-mds[65258]: -- 172.21.15.100:0/3470128856 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5612e8883a40 con 0x5612e8857000 2024-03-21T08:54:32.267 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:32 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k[65254]: starting mds.k at 2024-03-21T08:54:32.267 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:32 smithi100 bash[65239]: 69d3153453cac6a2c8a0eb7fa5cc380f01d21d9a535f1b204682f7384b3f7e03 2024-03-21T08:54:32.268 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 08:54:32 smithi100 systemd[1]: Started Ceph mds.k for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:32.383 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.387+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad8c1a0250 con 0x7fad8c100f40 2024-03-21T08:54:32.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.387+0000 7fad7b7fe640 1 -- 172.21.15.87:0/1574417541 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1325 (secure 0 0 0) 0x7fad7c09b5f0 con 0x7fad8c100f40 2024-03-21T08:54:32.384 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:32.384 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:54:32.384 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 2 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad740a2f20 msgr2=0x7fad740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad740a2f20 0x7fad740a53e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fad8002c7b0 tx=0x7fad80005fd0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 msgr2=0x7fad8c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 0x7fad8c197f80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fad7c00d770 tx=0x7fad7c00dc30 comp rx=0 tx=0).stop 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 shutdown_connections 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad740a2f20 0x7fad740a53e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fad8c101900 0x7fad8c19eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fad8c100f40 0x7fad8c197f80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 --2- 172.21.15.87:0/1574417541 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fad8c0ffd50 0x7fad8c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:32.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 >> 172.21.15.87:0/1574417541 conn(0x7fad8c0fb2c0 msgr2=0x7fad8c0fcdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:32.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 shutdown_connections 2024-03-21T08:54:32.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:32.390+0000 7fad9084b640 1 -- 172.21.15.87:0/1574417541 wait complete. 2024-03-21T08:54:32.396 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:31 smithi087 ceph-mon[32385]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:33.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.525 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: Deploying daemon mds.j on smithi087 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1574417541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6832/1637520984,v1:172.21.15.100:6833/1637520984] up:boot 2024-03-21T08:54:33.526 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: fsmap 2 up:standby 2024-03-21T08:54:33.527 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: Deploying daemon mds.j on smithi087 2024-03-21T08:54:33.528 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1574417541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6832/1637520984,v1:172.21.15.100:6833/1637520984] up:boot 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: fsmap 2 up:standby 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:33.529 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:33.530 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: Deploying daemon mds.j on smithi087 2024-03-21T08:54:33.530 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1574417541' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:33.530 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:33.530 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6832/1637520984,v1:172.21.15.100:6833/1637520984] up:boot 2024-03-21T08:54:33.530 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: fsmap 2 up:standby 2024-03-21T08:54:33.530 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T08:54:34.398 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 -- 172.21.15.87:0/1137469137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 msgr2=0x7f33c0101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 --2- 172.21.15.87:0/1137469137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0101af0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f33b0009900 tx=0x7f33b002bc50 comp rx=0 tx=0).stop 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 -- 172.21.15.87:0/1137469137 shutdown_connections 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 --2- 172.21.15.87:0/1137469137 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33c0103280 0x7f33c0109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 --2- 172.21.15.87:0/1137469137 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c01028c0 0x7f33c0102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 --2- 172.21.15.87:0/1137469137 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0101af0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 -- 172.21.15.87:0/1137469137 >> 172.21.15.87:0/1137469137 conn(0x7f33c00fcc40 msgr2=0x7f33c00ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:34.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 -- 172.21.15.87:0/1137469137 shutdown_connections 2024-03-21T08:54:34.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.515+0000 7f33c569a640 1 -- 172.21.15.87:0/1137469137 wait complete. 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 Processor -- start 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 -- start start 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0070030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33c01028c0 0x7f33c0070570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0103280 0x7f33c01a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010d370 con 0x7f33c01016d0 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010d1f0 con 0x7f33c0103280 2024-03-21T08:54:34.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.516+0000 7f33c569a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c010d4f0 con 0x7f33c01028c0 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0070030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0070030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54930/0 (socket says 172.21.15.87:54930) 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 -- 172.21.15.87:0/3184665474 learned_addr learned my addr 172.21.15.87:0/3184665474 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33be7fc640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33c01028c0 0x7f33c0070570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33bf7fe640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0103280 0x7f33c01a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 -- 172.21.15.87:0/3184665474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33c01028c0 msgr2=0x7f33c0070570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33c01028c0 0x7f33c0070570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 -- 172.21.15.87:0/3184665474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0103280 msgr2=0x7f33c01a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:34.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0103280 0x7f33c01a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 -- 172.21.15.87:0/3184665474 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c01a7fc0 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33beffd640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0070030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005ab0 tx=0x7f33b002c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0041070 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c01a8190 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33b0004450 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b0034440 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.517+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c01a8650 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.519+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c01089f0 con 0x7f33c01016d0 2024-03-21T08:54:34.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.519+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f33b0044070 con 0x7f33c01016d0 2024-03-21T08:54:34.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.520+0000 7f33a7fff640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33a80a2f20 0x7f33a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:34.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.520+0000 7f33be7fc640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33a80a2f20 0x7f33a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:34.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.520+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f33b00f0b10 con 0x7f33c01016d0 2024-03-21T08:54:34.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.521+0000 7f33be7fc640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33a80a2f20 0x7f33a80a53e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f33c0071540 tx=0x7f33b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:34.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.524+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33b00bdde0 con 0x7f33c01016d0 2024-03-21T08:54:34.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.746+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33c002d240 con 0x7f33c01016d0 2024-03-21T08:54:34.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.746+0000 7f33a7fff640 1 -- 172.21.15.87:0/3184665474 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2119 (secure 0 0 0) 0x7f33b00bd6e0 con 0x7f33c01016d0 2024-03-21T08:54:34.743 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:34.743 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:54:34.743 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 3 2024-03-21T08:54:34.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33a80a2f20 msgr2=0x7f33a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33a80a2f20 0x7f33a80a53e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f33c0071540 tx=0x7f33b400b410 comp rx=0 tx=0).stop 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 msgr2=0x7f33c0070030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0070030 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f33b0005ab0 tx=0x7f33b002c6e0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 shutdown_connections 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33a80a2f20 0x7f33a80a53e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33c0103280 0x7f33c01a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33c01028c0 0x7f33c0070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 --2- 172.21.15.87:0/3184665474 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33c01016d0 0x7f33c0070030 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 >> 172.21.15.87:0/3184665474 conn(0x7f33c00fcc40 msgr2=0x7f33c0105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 shutdown_connections 2024-03-21T08:54:34.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:34.748+0000 7f33c569a640 1 -- 172.21.15.87:0/3184665474 wait complete. 2024-03-21T08:54:34.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:34 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3184665474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:35.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:34 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3184665474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:35.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:34 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3184665474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:35.406 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 08:54:35 smithi087 systemd[1]: Started Ceph mds.j for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:35.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:35.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:35.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:35.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:35.970 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:35.971 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:35.971 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:35 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:36.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:35 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:36.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:35 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:36.757 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:36.866 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 -- 172.21.15.87:0/1071469868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd8102170 msgr2=0x7f5fd8108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 --2- 172.21.15.87:0/1071469868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd8102170 0x7f5fd8108a00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f5fd0009ca0 tx=0x7f5fd002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 -- 172.21.15.87:0/1071469868 shutdown_connections 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 --2- 172.21.15.87:0/1071469868 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd8102170 0x7f5fd8108a00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 --2- 172.21.15.87:0/1071469868 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fd81017b0 0x7f5fd8101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 --2- 172.21.15.87:0/1071469868 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd81005c0 0x7f5fd81009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 -- 172.21.15.87:0/1071469868 >> 172.21.15.87:0/1071469868 conn(0x7f5fd80fbb50 msgr2=0x7f5fd80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 -- 172.21.15.87:0/1071469868 shutdown_connections 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 -- 172.21.15.87:0/1071469868 wait complete. 2024-03-21T08:54:36.867 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.870+0000 7f5fddca7640 1 Processor -- start 2024-03-21T08:54:36.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 -- start start 2024-03-21T08:54:36.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 0x7f5fd8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:36.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd81017b0 0x7f5fd8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:36.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fd8102170 0x7f5fd819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:36.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810d3e0 con 0x7f5fd81005c0 2024-03-21T08:54:36.870 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810d260 con 0x7f5fd81017b0 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fddca7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd810d560 con 0x7f5fd8102170 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 0x7f5fd8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fd8102170 0x7f5fd819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd6ffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd81017b0 0x7f5fd8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 0x7f5fd8197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54982/0 (socket says 172.21.15.87:54982) 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd77fe640 1 -- 172.21.15.87:0/2312419850 learned_addr learned my addr 172.21.15.87:0/2312419850 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd77fe640 1 -- 172.21.15.87:0/2312419850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fd8102170 msgr2=0x7f5fd819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.871+0000 7f5fd77fe640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fd8102170 0x7f5fd819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd77fe640 1 -- 172.21.15.87:0/2312419850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd81017b0 msgr2=0x7f5fd8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd77fe640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd81017b0 0x7f5fd8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd77fe640 1 -- 172.21.15.87:0/2312419850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd819f620 con 0x7f5fd81005c0 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd77fe640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 0x7f5fd8197a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b450 tx=0x7f5fc800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:36.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8015020 con 0x7f5fd81005c0 2024-03-21T08:54:36.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd819f910 con 0x7f5fd81005c0 2024-03-21T08:54:36.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5fc8011d70 con 0x7f5fd81005c0 2024-03-21T08:54:36.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc800f340 con 0x7f5fd81005c0 2024-03-21T08:54:36.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.872+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd819feb0 con 0x7f5fd81005c0 2024-03-21T08:54:36.872 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.873+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd81078e0 con 0x7f5fd81005c0 2024-03-21T08:54:36.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.874+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5fc8018020 con 0x7f5fd81005c0 2024-03-21T08:54:36.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.875+0000 7f5fd4ff9640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5fc00a2f20 0x7f5fc00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:36.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.875+0000 7f5fd6ffd640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5fc00a2f20 0x7f5fc00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:36.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.875+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5fc80ce8f0 con 0x7f5fd81005c0 2024-03-21T08:54:36.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.875+0000 7f5fd6ffd640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5fc00a2f20 0x7f5fc00a53e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8198f10 tx=0x7f5fcc00a250 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:36.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:36.879+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5fc801f020 con 0x7f5fd81005c0 2024-03-21T08:54:36.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:36 smithi105 ceph-mon[35993]: Deploying daemon mds.d on smithi087 2024-03-21T08:54:36.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:36 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.87:6834/2095083606,v1:172.21.15.87:6835/2095083606] up:boot 2024-03-21T08:54:36.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:36 smithi105 ceph-mon[35993]: fsmap 3 up:standby 2024-03-21T08:54:36.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:36 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:54:37.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.111+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5fd802d200 con 0x7f5fd81005c0 2024-03-21T08:54:37.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.112+0000 7f5fd4ff9640 1 -- 172.21.15.87:0/2312419850 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2910 (secure 0 0 0) 0x7f5fc800d040 con 0x7f5fd81005c0 2024-03-21T08:54:37.108 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:37.108 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:54:37.108 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 4 2024-03-21T08:54:37.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5fc00a2f20 msgr2=0x7f5fc00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:37.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5fc00a2f20 0x7f5fc00a53e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8198f10 tx=0x7f5fcc00a250 comp rx=0 tx=0).stop 2024-03-21T08:54:37.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 msgr2=0x7f5fd8197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:37.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 0x7f5fd8197a00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800b450 tx=0x7f5fc800b910 comp rx=0 tx=0).stop 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 shutdown_connections 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5fc00a2f20 0x7f5fc00a53e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5fd8102170 0x7f5fd819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5fd81017b0 0x7f5fd8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 --2- 172.21.15.87:0/2312419850 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5fd81005c0 0x7f5fd8197a00 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 >> 172.21.15.87:0/2312419850 conn(0x7f5fd80fbb50 msgr2=0x7f5fd8104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 shutdown_connections 2024-03-21T08:54:37.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:37.114+0000 7f5fddca7640 1 -- 172.21.15.87:0/2312419850 wait complete. 2024-03-21T08:54:37.120 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:36 smithi087 ceph-mon[32385]: Deploying daemon mds.d on smithi087 2024-03-21T08:54:37.120 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:36 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.87:6834/2095083606,v1:172.21.15.87:6835/2095083606] up:boot 2024-03-21T08:54:37.120 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:36 smithi087 ceph-mon[32385]: fsmap 3 up:standby 2024-03-21T08:54:37.120 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:54:37.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:36 smithi100 ceph-mon[36306]: Deploying daemon mds.d on smithi087 2024-03-21T08:54:37.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:36 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.87:6834/2095083606,v1:172.21.15.87:6835/2095083606] up:boot 2024-03-21T08:54:37.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:36 smithi100 ceph-mon[36306]: fsmap 3 up:standby 2024-03-21T08:54:37.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:36 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-03-21T08:54:37.407 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 systemd[1]: Starting Ceph mds.d for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:54:37.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:37 smithi087 ceph-mon[32385]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:37.813 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:37 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2312419850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:37.813 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 podman[72728]: 2024-03-21 08:54:37.464943716 +0000 UTC m=+0.018850415 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:54:37.814 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 podman[72728]: 2024-03-21 08:54:37.582643147 +0000 UTC m=+0.136549882 container create 11bdb4052c12681f226536db9a9f321d4a304921977d704c99c882c4f01ed492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:54:37.814 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 podman[72728]: 2024-03-21 08:54:37.818451872 +0000 UTC m=+0.372358575 container init 11bdb4052c12681f226536db9a9f321d4a304921977d704c99c882c4f01ed492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True) 2024-03-21T08:54:38.157 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 podman[72728]: 2024-03-21 08:54:37.821367725 +0000 UTC m=+0.375274425 container start 11bdb4052c12681f226536db9a9f321d4a304921977d704c99c882c4f01ed492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T08:54:38.157 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 ceph-mds[72747]: -- 172.21.15.87:0/4286941394 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b3aea4ba40 con 0x55b3aea1e800 2024-03-21T08:54:38.157 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d[72743]: starting mds.d at 2024-03-21T08:54:38.157 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 bash[72728]: 11bdb4052c12681f226536db9a9f321d4a304921977d704c99c882c4f01ed492 2024-03-21T08:54:38.157 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 08:54:37 smithi087 systemd[1]: Started Ceph mds.d for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:38.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:37 smithi105 ceph-mon[35993]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:38.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:37 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2312419850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:38.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:37 smithi100 ceph-mon[36306]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:38.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:37 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2312419850' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:39.123 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:39.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: Deploying daemon mds.e on smithi100 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.87:6836/4247247896,v1:172.21.15.87:6837/4247247896] up:boot 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: fsmap 4 up:standby 2024-03-21T08:54:39.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:38 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 -- 172.21.15.87:0/622184980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770103340 msgr2=0x7f1770109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 --2- 172.21.15.87:0/622184980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770103340 0x7f1770109bd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1768009cd0 tx=0x7f176802c0f0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 -- 172.21.15.87:0/622184980 shutdown_connections 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 --2- 172.21.15.87:0/622184980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770103340 0x7f1770109bd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 --2- 172.21.15.87:0/622184980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1770102980 0x7f1770102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 --2- 172.21.15.87:0/622184980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17700ff9b0 0x7f17700ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 -- 172.21.15.87:0/622184980 >> 172.21.15.87:0/622184980 conn(0x7f17700fb2e0 msgr2=0x7f17700fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:39.240 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 -- 172.21.15.87:0/622184980 shutdown_connections 2024-03-21T08:54:39.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.243+0000 7f1775027640 1 -- 172.21.15.87:0/622184980 wait complete. 2024-03-21T08:54:39.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 Processor -- start 2024-03-21T08:54:39.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 -- start start 2024-03-21T08:54:39.241 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17700ff9b0 0x7f17701979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:39.242 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 0x7f1770197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1770103340 0x7f177019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177010d390 con 0x7f1770102980 2024-03-21T08:54:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177010d210 con 0x7f17700ff9b0 2024-03-21T08:54:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f1775027640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177010d510 con 0x7f1770103340 2024-03-21T08:54:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f176e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 0x7f1770197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:39.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f176e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 0x7f1770197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43374/0 (socket says 172.21.15.87:43374) 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.244+0000 7f176e59c640 1 -- 172.21.15.87:0/3124462260 learned_addr learned my addr 172.21.15.87:0/3124462260 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176f59e640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1770103340 0x7f177019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176ed9d640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17700ff9b0 0x7f17701979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176e59c640 1 -- 172.21.15.87:0/3124462260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1770103340 msgr2=0x7f177019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176e59c640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1770103340 0x7f177019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176e59c640 1 -- 172.21.15.87:0/3124462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17700ff9b0 msgr2=0x7f17701979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:39.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176e59c640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17700ff9b0 0x7f17701979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176e59c640 1 -- 172.21.15.87:0/3124462260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f177019f5f0 con 0x7f1770102980 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f176e59c640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 0x7f1770197f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f176400d770 tx=0x7f176400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764017070 con 0x7f1770102980 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1764005500 con 0x7f1770102980 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1764004e00 con 0x7f1770102980 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f177019f8e0 con 0x7f1770102980 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.245+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177019fde0 con 0x7f1770102980 2024-03-21T08:54:39.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.246+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1770108ab0 con 0x7f1770102980 2024-03-21T08:54:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.247+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1764004fa0 con 0x7f1770102980 2024-03-21T08:54:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.248+0000 7f1757fff640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17580a2f20 0x7f17580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.248+0000 7f176ed9d640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17580a2f20 0x7f17580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.248+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f17640ce9a0 con 0x7f1770102980 2024-03-21T08:54:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.248+0000 7f176ed9d640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17580a2f20 0x7f17580a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1770069af0 tx=0x7f1760005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:39.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.252+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f176409bc70 con 0x7f1770102980 2024-03-21T08:54:39.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: Deploying daemon mds.e on smithi100 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.87:6836/4247247896,v1:172.21.15.87:6837/4247247896] up:boot 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: fsmap 4 up:standby 2024-03-21T08:54:39.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:38 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:54:39.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: Deploying daemon mds.e on smithi100 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.87:6836/4247247896,v1:172.21.15.87:6837/4247247896] up:boot 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: fsmap 4 up:standby 2024-03-21T08:54:39.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:38 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-03-21T08:54:39.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.466+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f177002d250 con 0x7f1770102980 2024-03-21T08:54:39.463 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.467+0000 7f1757fff640 1 -- 172.21.15.87:0/3124462260 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3701 (secure 0 0 0) 0x7f176409b570 con 0x7f1770102980 2024-03-21T08:54:39.463 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:39.463 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2024-03-21T08:54:39.464 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 5 2024-03-21T08:54:39.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17580a2f20 msgr2=0x7f17580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17580a2f20 0x7f17580a53e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1770069af0 tx=0x7f1760005f90 comp rx=0 tx=0).stop 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 msgr2=0x7f1770197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 0x7f1770197f10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f176400d770 tx=0x7f176400dc30 comp rx=0 tx=0).stop 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 shutdown_connections 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17580a2f20 0x7f17580a53e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1770103340 0x7f177019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1770102980 0x7f1770197f10 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 --2- 172.21.15.87:0/3124462260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17700ff9b0 0x7f17701979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:39.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 >> 172.21.15.87:0/3124462260 conn(0x7f17700fb2e0 msgr2=0x7f1770105c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:39.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 shutdown_connections 2024-03-21T08:54:39.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:39.469+0000 7f1775027640 1 -- 172.21.15.87:0/3124462260 wait complete. 2024-03-21T08:54:40.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:39 smithi100 ceph-mon[36306]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:40.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:39 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3124462260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:40.268 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 systemd[1]: Starting Ceph mds.e for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:54:40.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:39 smithi087 ceph-mon[32385]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:40.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:39 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3124462260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:40.407 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:39 smithi105 ceph-mon[35993]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:40.407 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:39 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3124462260' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:40.697 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 podman[65558]: 2024-03-21 08:54:40.342139347 +0000 UTC m=+0.017747604 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:54:40.697 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 podman[65558]: 2024-03-21 08:54:40.465621892 +0000 UTC m=+0.141230133 container create ae2d741befc59e7d84e3229b7292bfb5888e6c65327f76b1efabace934179ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0) 2024-03-21T08:54:41.017 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 podman[65558]: 2024-03-21 08:54:40.701731549 +0000 UTC m=+0.377339802 container init ae2d741befc59e7d84e3229b7292bfb5888e6c65327f76b1efabace934179ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T08:54:41.017 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 podman[65558]: 2024-03-21 08:54:40.704333589 +0000 UTC m=+0.379941822 container start ae2d741befc59e7d84e3229b7292bfb5888e6c65327f76b1efabace934179ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-03-21T08:54:41.017 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 ceph-mds[65577]: -- 172.21.15.100:0/2881737867 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x565050029a40 con 0x56504fffc800 2024-03-21T08:54:41.017 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e[65573]: starting mds.e at 2024-03-21T08:54:41.017 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 bash[65558]: ae2d741befc59e7d84e3229b7292bfb5888e6c65327f76b1efabace934179ac0 2024-03-21T08:54:41.018 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 08:54:40 smithi100 systemd[1]: Started Ceph mds.e for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:41.478 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:41.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 -- 172.21.15.87:0/3001400110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe478101270 msgr2=0x7fe478108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:41.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 --2- 172.21.15.87:0/3001400110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe478101270 0x7fe478108b10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe470009ca0 tx=0x7fe47002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 -- 172.21.15.87:0/3001400110 shutdown_connections 2024-03-21T08:54:41.598 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 --2- 172.21.15.87:0/3001400110 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe478101270 0x7fe478108b10 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 --2- 172.21.15.87:0/3001400110 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe4781008b0 0x7fe478100d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 --2- 172.21.15.87:0/3001400110 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 0x7fe478101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 -- 172.21.15.87:0/3001400110 >> 172.21.15.87:0/3001400110 conn(0x7fe4780fb2e0 msgr2=0x7fe4780fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 -- 172.21.15.87:0/3001400110 shutdown_connections 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.594+0000 7fe47f13b640 1 -- 172.21.15.87:0/3001400110 wait complete. 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 Processor -- start 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 -- start start 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 0x7fe4781979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe478101270 0x7fe478197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 0x7fe47819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47810d360 con 0x7fe4781008b0 2024-03-21T08:54:41.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47810d1e0 con 0x7fe478101270 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47810d4e0 con 0x7fe478101b80 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 0x7fe4781979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 0x7fe4781979a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43402/0 (socket says 172.21.15.87:43402) 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 -- 172.21.15.87:0/1806247694 learned_addr learned my addr 172.21.15.87:0/1806247694 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe477fff640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe478101270 0x7fe478197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47d6d8640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 0x7fe47819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 -- 172.21.15.87:0/1806247694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 msgr2=0x7fe47819ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 0x7fe47819ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 -- 172.21.15.87:0/1806247694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe478101270 msgr2=0x7fe478197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe478101270 0x7fe478197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 -- 172.21.15.87:0/1806247694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47819f5c0 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47d6d8640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 0x7fe47819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe477fff640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe478101270 0x7fe478197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47ced7640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 0x7fe4781979a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00d950 tx=0x7fe46c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c017020 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47819f8b0 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe46c009c30 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.595+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe46c00f6b0 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.596+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47819fe50 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.597+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4780688e0 con 0x7fe4781008b0 2024-03-21T08:54:41.601 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.597+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe46c01a020 con 0x7fe4781008b0 2024-03-21T08:54:41.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.598+0000 7fe475ffb640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4680a2f20 0x7fe4680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:41.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.598+0000 7fe477fff640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4680a2f20 0x7fe4680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:41.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.598+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe46c0cef30 con 0x7fe4781008b0 2024-03-21T08:54:41.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.598+0000 7fe477fff640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4680a2f20 0x7fe4680a53e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe478198eb0 tx=0x7fe464005ec0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:41.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.603+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe46c021020 con 0x7fe4781008b0 2024-03-21T08:54:41.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.814+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe47802d180 con 0x7fe4781008b0 2024-03-21T08:54:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.814+0000 7fe475ffb640 1 -- 172.21.15.87:0/1806247694 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4495 (secure 0 0 0) 0x7fe46c09c200 con 0x7fe4781008b0 2024-03-21T08:54:41.811 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:41.811 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6}],"filesystems":[]} 2024-03-21T08:54:41.811 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 6 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4680a2f20 msgr2=0x7fe4680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4680a2f20 0x7fe4680a53e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe478198eb0 tx=0x7fe464005ec0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 msgr2=0x7fe4781979a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 0x7fe4781979a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00d950 tx=0x7fe46c00de10 comp rx=0 tx=0).stop 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 shutdown_connections 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4680a2f20 0x7fe4680a53e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe478101b80 0x7fe47819ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe478101270 0x7fe478197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 --2- 172.21.15.87:0/1806247694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4781008b0 0x7fe4781979a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:41.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.816+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 >> 172.21.15.87:0/1806247694 conn(0x7fe4780fb2e0 msgr2=0x7fe478104b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:41.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.817+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 shutdown_connections 2024-03-21T08:54:41.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:41.817+0000 7fe47f13b640 1 -- 172.21.15.87:0/1806247694 wait complete. 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: Deploying daemon mds.f on smithi105 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] up:boot 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: fsmap 5 up:standby 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:54:41.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:41 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1806247694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:42.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: Deploying daemon mds.f on smithi105 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] up:boot 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: fsmap 5 up:standby 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:54:42.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:41 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1806247694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:42.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: Deploying daemon mds.f on smithi105 2024-03-21T08:54:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] up:boot 2024-03-21T08:54:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: fsmap 5 up:standby 2024-03-21T08:54:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T08:54:42.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:41 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1806247694' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:43.234 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 systemd[1]: Starting Ceph mds.f for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:54:43.530 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 podman[66617]: 2024-03-21 08:54:43.184227308 +0000 UTC m=+0.017775354 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:54:43.531 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 podman[66617]: 2024-03-21 08:54:43.299784876 +0000 UTC m=+0.133332961 container create 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T08:54:43.531 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 podman[66617]: 2024-03-21 08:54:43.535849144 +0000 UTC m=+0.369397202 container init 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0) 2024-03-21T08:54:43.825 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:43.848 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 podman[66617]: 2024-03-21 08:54:43.538446861 +0000 UTC m=+0.371994901 container start 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-03-21T08:54:43.848 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mds[66636]: -- 172.21.15.105:0/2286408765 <== mon.1 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5561a140da40 con 0x5561a13e0800 2024-03-21T08:54:43.848 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f[66632]: starting mds.f at 2024-03-21T08:54:43.848 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 bash[66617]: 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 2024-03-21T08:54:43.848 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 08:54:43 smithi105 systemd[1]: Started Ceph mds.f for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:43.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 -- 172.21.15.87:0/4138262267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c102980 msgr2=0x7f1f2c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 --2- 172.21.15.87:0/4138262267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c102980 0x7f1f2c102e00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1f20009930 tx=0x7f1f2002bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 -- 172.21.15.87:0/4138262267 shutdown_connections 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 --2- 172.21.15.87:0/4138262267 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f2c103340 0x7f1f2c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 --2- 172.21.15.87:0/4138262267 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c102980 0x7f1f2c102e00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 --2- 172.21.15.87:0/4138262267 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c0ff9b0 0x7f1f2c0ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 -- 172.21.15.87:0/4138262267 >> 172.21.15.87:0/4138262267 conn(0x7f1f2c0fb2e0 msgr2=0x7f1f2c0fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 -- 172.21.15.87:0/4138262267 shutdown_connections 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 -- 172.21.15.87:0/4138262267 wait complete. 2024-03-21T08:54:43.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.942+0000 7f1f31405640 1 Processor -- start 2024-03-21T08:54:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 -- start start 2024-03-21T08:54:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f2c0ff9b0 0x7f1f2c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:43.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102980 0x7f1f2c197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 0x7f1f2c19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c10d390 con 0x7f1f2c103340 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c10d210 con 0x7f1f2c102980 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f31405640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f2c10d510 con 0x7f1f2c0ff9b0 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 0x7f1f2c19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 0x7f1f2c19efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43416/0 (socket says 172.21.15.87:43416) 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 -- 172.21.15.87:0/633177749 learned_addr learned my addr 172.21.15.87:0/633177749 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2affd640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f2c0ff9b0 0x7f1f2c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2a7fc640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102980 0x7f1f2c197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 -- 172.21.15.87:0/633177749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f2c0ff9b0 msgr2=0x7f1f2c197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f2c0ff9b0 0x7f1f2c197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 -- 172.21.15.87:0/633177749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102980 msgr2=0x7f1f2c197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:43.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102980 0x7f1f2c197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 -- 172.21.15.87:0/633177749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2c19f650 con 0x7f1f2c103340 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2a7fc640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102980 0x7f1f2c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.943+0000 7f1f2b7fe640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 0x7f1f2c19efe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400eb10 tx=0x7f1f2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.944+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24004b20 con 0x7f1f2c103340 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.944+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f24004cc0 con 0x7f1f2c103340 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.944+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f2c19f940 con 0x7f1f2c103340 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.944+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f24010760 con 0x7f1f2c103340 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.944+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f2c19fe60 con 0x7f1f2c103340 2024-03-21T08:54:43.943 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.945+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f2c108ba0 con 0x7f1f2c103340 2024-03-21T08:54:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.946+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1f24005000 con 0x7f1f2c103340 2024-03-21T08:54:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.946+0000 7f1f13fff640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f140a2f20 0x7f1f140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.946+0000 7f1f2affd640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f140a2f20 0x7f1f140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.947+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1f240cfa20 con 0x7f1f2c103340 2024-03-21T08:54:43.944 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.947+0000 7f1f2affd640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f140a2f20 0x7f1f140a53e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c069af0 tx=0x7f1f1c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:43.946 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:43.950+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1f2409ccf0 con 0x7f1f2c103340 2024-03-21T08:54:44.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:44.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:44.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:44.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.164+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1f2c02d070 con 0x7f1f2c103340 2024-03-21T08:54:44.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.164+0000 7f1f13fff640 1 -- 172.21.15.87:0/633177749 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f1f2409c5f0 con 0x7f1f2c103340 2024-03-21T08:54:44.161 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:44.161 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-21T08:54:44.161 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 7 2024-03-21T08:54:44.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.166+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f140a2f20 msgr2=0x7f1f140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:44.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.166+0000 7f1f31405640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f140a2f20 0x7f1f140a53e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1f2c069af0 tx=0x7f1f1c00b390 comp rx=0 tx=0).stop 2024-03-21T08:54:44.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.166+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 msgr2=0x7f1f2c19efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:44.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.166+0000 7f1f31405640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 0x7f1f2c19efe0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1f2400eb10 tx=0x7f1f2400c2d0 comp rx=0 tx=0).stop 2024-03-21T08:54:44.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.166+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 shutdown_connections 2024-03-21T08:54:44.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f140a2f20 0x7f1f140a53e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:44.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f2c103340 0x7f1f2c19efe0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:44.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f2c102980 0x7f1f2c197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:44.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 --2- 172.21.15.87:0/633177749 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f2c0ff9b0 0x7f1f2c197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:44.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 >> 172.21.15.87:0/633177749 conn(0x7f1f2c0fb2e0 msgr2=0x7f1f2c105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:44.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 shutdown_connections 2024-03-21T08:54:44.164 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:44.167+0000 7f1f31405640 1 -- 172.21.15.87:0/633177749 wait complete. 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:44.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:44.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:44 smithi087 ceph-mon[32385]: Deploying daemon mds.c on smithi105 2024-03-21T08:54:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] up:boot 2024-03-21T08:54:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:44 smithi087 ceph-mon[32385]: fsmap 6 up:standby 2024-03-21T08:54:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:54:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:44 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/633177749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:44 smithi105 ceph-mon[35993]: Deploying daemon mds.c on smithi105 2024-03-21T08:54:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] up:boot 2024-03-21T08:54:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:44 smithi105 ceph-mon[35993]: fsmap 6 up:standby 2024-03-21T08:54:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:54:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:44 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/633177749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:45.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:44 smithi100 ceph-mon[36306]: Deploying daemon mds.c on smithi105 2024-03-21T08:54:45.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] up:boot 2024-03-21T08:54:45.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:44 smithi100 ceph-mon[36306]: fsmap 6 up:standby 2024-03-21T08:54:45.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T08:54:45.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:44 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/633177749' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:45.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:45 smithi105 ceph-mon[35993]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:46.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:45 smithi100 ceph-mon[36306]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:46.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:45 smithi087 ceph-mon[32385]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:46.175 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:46.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2197305511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00fff30 msgr2=0x7f2ad0102360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:46.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2197305511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00fff30 0x7f2ad0102360 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2acc009900 tx=0x7f2acc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:46.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2197305511 shutdown_connections 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2197305511 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 0x7f2ad01056d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2197305511 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad01028a0 0x7f2ad0104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2197305511 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad00fff30 0x7f2ad0102360 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2197305511 >> 172.21.15.87:0/2197305511 conn(0x7f2ad00fbb30 msgr2=0x7f2ad00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2197305511 shutdown_connections 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.292+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2197305511 wait complete. 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 Processor -- start 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 -- start start 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad00fff30 0x7f2ad019e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 0x7f2ad019e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 0x7f2ad01a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0113930 con 0x7f2ad01028a0 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad01137b0 con 0x7f2ad00fff30 2024-03-21T08:54:46.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad76e8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad0113ab0 con 0x7f2ad0105250 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 0x7f2ad019e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 0x7f2ad019e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:43436/0 (socket says 172.21.15.87:43436) 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 -- 172.21.15.87:0/2242024277 learned_addr learned my addr 172.21.15.87:0/2242024277 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad5c85640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 0x7f2ad01a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad5484640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad00fff30 0x7f2ad019e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 -- 172.21.15.87:0/2242024277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 msgr2=0x7f2ad01a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 0x7f2ad01a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 -- 172.21.15.87:0/2242024277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad00fff30 msgr2=0x7f2ad019e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.293+0000 7f2ad4c83640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad00fff30 0x7f2ad019e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.292 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ad4c83640 1 -- 172.21.15.87:0/2242024277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad01a5c10 con 0x7f2ad01028a0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ad5c85640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 0x7f2ad01a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ad4c83640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 0x7f2ad019e580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d950 tx=0x7f2ac000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac0017030 con 0x7f2ad01028a0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2ac0004020 con 0x7f2ad01028a0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ad01a5f00 con 0x7f2ad01028a0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ac00047a0 con 0x7f2ad01028a0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.294+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ad01a6410 con 0x7f2ad01028a0 2024-03-21T08:54:46.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.295+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ad00fd620 con 0x7f2ad01028a0 2024-03-21T08:54:46.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.296+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2ac00041d0 con 0x7f2ad01028a0 2024-03-21T08:54:46.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.297+0000 7f2ac67fc640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2abc0a2fa0 0x7f2abc0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:46.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.297+0000 7f2ad5484640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2abc0a2fa0 0x7f2abc0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:46.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.297+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2ac00ceee0 con 0x7f2ad01028a0 2024-03-21T08:54:46.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.297+0000 7f2ad5484640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2abc0a2fa0 0x7f2abc0a5460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02c7b0 tx=0x7f2acc005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:46.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.300+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2ac009c230 con 0x7f2ad01028a0 2024-03-21T08:54:46.303 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 08:54:46 smithi105 podman[66926]: 2024-03-21 08:54:46.285795201 +0000 UTC m=+0.349069101 container start bc8bc523e9fb99869075139b2eacadb14a0b84b2e1489b07edd728343bbb52ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-03-21T08:54:46.304 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 08:54:46 smithi105 ceph-mds[66945]: -- 172.21.15.105:0/4037421140 <== mon.1 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561de9529a40 con 0x561de94fc800 2024-03-21T08:54:46.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.519+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ad002d070 con 0x7f2ad01028a0 2024-03-21T08:54:46.514 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.519+0000 7f2ac67fc640 1 -- 172.21.15.87:0/2242024277 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5289 (secure 0 0 0) 0x7f2ac009bb30 con 0x7f2ad01028a0 2024-03-21T08:54:46.515 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:46.515 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7}],"filesystems":[]} 2024-03-21T08:54:46.515 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 7 2024-03-21T08:54:46.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2abc0a2fa0 msgr2=0x7f2abc0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2abc0a2fa0 0x7f2abc0a5460 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2acc02c7b0 tx=0x7f2acc005fd0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 msgr2=0x7f2ad019e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 0x7f2ad019e580 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2ac000d950 tx=0x7f2ac000de10 comp rx=0 tx=0).stop 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 shutdown_connections 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2abc0a2fa0 0x7f2abc0a5460 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2ad0105250 0x7f2ad01a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2ad01028a0 0x7f2ad019e580 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 --2- 172.21.15.87:0/2242024277 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2ad00fff30 0x7f2ad019e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 >> 172.21.15.87:0/2242024277 conn(0x7f2ad00fbb30 msgr2=0x7f2ad0103250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 shutdown_connections 2024-03-21T08:54:46.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:46.521+0000 7f2ad76e8640 1 -- 172.21.15.87:0/2242024277 wait complete. 2024-03-21T08:54:46.733 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 08:54:46 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c[66941]: starting mds.c at 2024-03-21T08:54:46.734 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 08:54:46 smithi105 bash[66926]: bc8bc523e9fb99869075139b2eacadb14a0b84b2e1489b07edd728343bbb52ff 2024-03-21T08:54:46.734 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 08:54:46 smithi105 systemd[1]: Started Ceph mds.c for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: Deploying daemon mds.h on smithi100 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2242024277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6836/2656551527,v1:172.21.15.105:6837/2656551527] up:boot 2024-03-21T08:54:47.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: fsmap 7 up:standby 2024-03-21T08:54:47.735 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: Deploying daemon mds.h on smithi100 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2242024277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:47.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6836/2656551527,v1:172.21.15.105:6837/2656551527] up:boot 2024-03-21T08:54:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: fsmap 7 up:standby 2024-03-21T08:54:47.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:54:47.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: Deploying daemon mds.h on smithi100 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2242024277' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6836/2656551527,v1:172.21.15.105:6837/2656551527] up:boot 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: fsmap 7 up:standby 2024-03-21T08:54:47.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T08:54:48.529 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:48.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 -- 172.21.15.87:0/3123492775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 msgr2=0x7f2928108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:48.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 --2- 172.21.15.87:0/3123492775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f2928108a00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2920009900 tx=0x7f292002bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:48.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 -- 172.21.15.87:0/3123492775 shutdown_connections 2024-03-21T08:54:48.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 --2- 172.21.15.87:0/3123492775 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f2928108a00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 --2- 172.21.15.87:0/3123492775 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29281017b0 0x7f2928101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 --2- 172.21.15.87:0/3123492775 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281005c0 0x7f29281009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.645+0000 7f292f5b9640 1 -- 172.21.15.87:0/3123492775 >> 172.21.15.87:0/3123492775 conn(0x7f29280fbb30 msgr2=0x7f29280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 -- 172.21.15.87:0/3123492775 shutdown_connections 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 -- 172.21.15.87:0/3123492775 wait complete. 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 Processor -- start 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 -- start start 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29281005c0 0x7f29280693d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281017b0 0x7f292819c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f292819ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292810c240 con 0x7f2928102170 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292810c0c0 con 0x7f29281005c0 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292f5b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292810c3c0 con 0x7f29281017b0 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292db56640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f292819ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:48.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292db56640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f292819ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47494/0 (socket says 172.21.15.87:47494) 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292db56640 1 -- 172.21.15.87:0/2874398640 learned_addr learned my addr 172.21.15.87:0/2874398640 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.646+0000 7f292d355640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29281005c0 0x7f29280693d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292cb54640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281017b0 0x7f292819c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292db56640 1 -- 172.21.15.87:0/2874398640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281017b0 msgr2=0x7f292819c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292db56640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281017b0 0x7f292819c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292db56640 1 -- 172.21.15.87:0/2874398640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29281005c0 msgr2=0x7f29280693d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292db56640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29281005c0 0x7f29280693d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292db56640 1 -- 172.21.15.87:0/2874398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292819f5d0 con 0x7f2928102170 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292cb54640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281017b0 0x7f292819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292db56640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f292819ed10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f292002c7b0 tx=0x7f2920005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920041070 con 0x7f2928102170 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292819f860 con 0x7f2928102170 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29200054e0 con 0x7f2928102170 2024-03-21T08:54:48.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920034440 con 0x7f2928102170 2024-03-21T08:54:48.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.647+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292819fd50 con 0x7f2928102170 2024-03-21T08:54:48.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.648+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29281078e0 con 0x7f2928102170 2024-03-21T08:54:48.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.649+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2920044070 con 0x7f2928102170 2024-03-21T08:54:48.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.650+0000 7f291e7fc640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29140a2f20 0x7f29140a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:48.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.650+0000 7f292d355640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29140a2f20 0x7f29140a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:48.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.650+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f29200f0f90 con 0x7f2928102170 2024-03-21T08:54:48.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.650+0000 7f292d355640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29140a2f20 0x7f29140a53e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2928101610 tx=0x7f2924009680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:48.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.653+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f29200be260 con 0x7f2928102170 2024-03-21T08:54:48.767 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:48 smithi100 systemd[1]: Starting Ceph mds.h for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:54:48.860 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.864+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f292802d230 con 0x7f2928102170 2024-03-21T08:54:48.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.865+0000 7f291e7fc640 1 -- 172.21.15.87:0/2874398640 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+6083 (secure 0 0 0) 0x7f292802d230 con 0x7f2928102170 2024-03-21T08:54:48.861 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:48.861 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-21T08:54:48.861 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 8 2024-03-21T08:54:48.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29140a2f20 msgr2=0x7f29140a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29140a2f20 0x7f29140a53e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2928101610 tx=0x7f2924009680 comp rx=0 tx=0).stop 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 msgr2=0x7f292819ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f292819ed10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f292002c7b0 tx=0x7f2920005dd0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 shutdown_connections 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29140a2f20 0x7f29140a53e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2928102170 0x7f292819ed10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29281017b0 0x7f292819c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 --2- 172.21.15.87:0/2874398640 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29281005c0 0x7f29280693d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:48.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 >> 172.21.15.87:0/2874398640 conn(0x7f29280fbb30 msgr2=0x7f2928104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:48.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 shutdown_connections 2024-03-21T08:54:48.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:48.867+0000 7f292f5b9640 1 -- 172.21.15.87:0/2874398640 wait complete. 2024-03-21T08:54:49.139 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:48 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2874398640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:49.139 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:48 smithi100 podman[65879]: 2024-03-21 08:54:48.763200742 +0000 UTC m=+0.017804448 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:54:49.139 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:48 smithi100 podman[65879]: 2024-03-21 08:54:48.889296165 +0000 UTC m=+0.143899857 container create fff9dbdc6e87c9bc16ede6f61fd5252812d7e46a71b3bd4b0d55d7c06a42c4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:54:49.139 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:49 smithi100 podman[65879]: 2024-03-21 08:54:49.143438646 +0000 UTC m=+0.398042357 container init fff9dbdc6e87c9bc16ede6f61fd5252812d7e46a71b3bd4b0d55d7c06a42c4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T08:54:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:48 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2874398640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:49.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:48 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2874398640' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:49.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:49 smithi100 podman[65879]: 2024-03-21 08:54:49.146140937 +0000 UTC m=+0.400744630 container start fff9dbdc6e87c9bc16ede6f61fd5252812d7e46a71b3bd4b0d55d7c06a42c4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:54:49.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mds[65897]: -- 172.21.15.100:0/783607396 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56063cca3a40 con 0x56063cc76800 2024-03-21T08:54:49.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h[65893]: starting mds.h at 2024-03-21T08:54:49.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:49 smithi100 bash[65879]: fff9dbdc6e87c9bc16ede6f61fd5252812d7e46a71b3bd4b0d55d7c06a42c4e9 2024-03-21T08:54:49.517 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 08:54:49 smithi100 systemd[1]: Started Ceph mds.h for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:50.173 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:50.173 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.173 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.173 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.174 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:50.174 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:50.174 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:49 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:50.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:50.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:49 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:50.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:49 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:50.874 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:50.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 -- 172.21.15.87:0/3076182720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81005e0 msgr2=0x7f00c8100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:50.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 --2- 172.21.15.87:0/3076182720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81005e0 0x7f00c8100a00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f00c4009900 tx=0x7f00c402bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:50.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 -- 172.21.15.87:0/3076182720 shutdown_connections 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 --2- 172.21.15.87:0/3076182720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 0x7f00c8108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 --2- 172.21.15.87:0/3076182720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00c81017d0 0x7f00c8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 --2- 172.21.15.87:0/3076182720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81005e0 0x7f00c8100a00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 -- 172.21.15.87:0/3076182720 >> 172.21.15.87:0/3076182720 conn(0x7f00c80fbb50 msgr2=0x7f00c80fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 -- 172.21.15.87:0/3076182720 shutdown_connections 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 -- 172.21.15.87:0/3076182720 wait complete. 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.994+0000 7f00cf612640 1 Processor -- start 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 -- start start 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00c81005e0 0x7f00c806a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 0x7f00c819eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 0x7f00c81a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c810c250 con 0x7f00c81017d0 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c810c0d0 con 0x7f00c8102190 2024-03-21T08:54:50.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00c810c3d0 con 0x7f00c81005e0 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 0x7f00c819eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 0x7f00c819eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47522/0 (socket says 172.21.15.87:47522) 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 -- 172.21.15.87:0/1546919371 learned_addr learned my addr 172.21.15.87:0/1546919371 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cd3ae640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00c81005e0 0x7f00c806a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cdbaf640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 0x7f00c81a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 -- 172.21.15.87:0/1546919371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00c81005e0 msgr2=0x7f00c806a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00c81005e0 0x7f00c806a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 -- 172.21.15.87:0/1546919371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 msgr2=0x7f00c81a10c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 0x7f00c81a10c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 -- 172.21.15.87:0/1546919371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c81a18b0 con 0x7f00c81017d0 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cdbaf640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 0x7f00c81a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:54:50.994 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ccbad640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 0x7f00c819eb70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d770 tx=0x7f00bc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc017070 con 0x7f00c81017d0 2024-03-21T08:54:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f00bc005500 con 0x7f00c81017d0 2024-03-21T08:54:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00c81a1b40 con 0x7f00c81017d0 2024-03-21T08:54:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.995+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00bc004e00 con 0x7f00c81017d0 2024-03-21T08:54:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.996+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00c81a2040 con 0x7f00c81017d0 2024-03-21T08:54:50.995 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.996+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00c8107900 con 0x7f00c81017d0 2024-03-21T08:54:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.998+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f00bc004fa0 con 0x7f00c81017d0 2024-03-21T08:54:50.996 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.998+0000 7f00ba7fc640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f00b40a2f50 0x7f00b40a5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:50.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.998+0000 7f00cd3ae640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f00b40a2f50 0x7f00b40a5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:50.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.998+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f00bc0ced30 con 0x7f00c81017d0 2024-03-21T08:54:50.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:50.999+0000 7f00cd3ae640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f00b40a2f50 0x7f00b40a5410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000c00 tx=0x7f00c4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:50.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.002+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f00bc09c000 con 0x7f00c81017d0 2024-03-21T08:54:51.082 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:50 smithi105 ceph-mon[35993]: Deploying daemon mds.i on smithi105 2024-03-21T08:54:51.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:50 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6836/2948361419,v1:172.21.15.100:6837/2948361419] up:boot 2024-03-21T08:54:51.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:50 smithi105 ceph-mon[35993]: fsmap 8 up:standby 2024-03-21T08:54:51.083 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:50 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:54:51.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:50 smithi087 ceph-mon[32385]: Deploying daemon mds.i on smithi105 2024-03-21T08:54:51.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:50 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6836/2948361419,v1:172.21.15.100:6837/2948361419] up:boot 2024-03-21T08:54:51.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:50 smithi087 ceph-mon[32385]: fsmap 8 up:standby 2024-03-21T08:54:51.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:50 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:54:51.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.217+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f00c802d070 con 0x7f00c81017d0 2024-03-21T08:54:51.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.217+0000 7f00ba7fc640 1 -- 172.21.15.87:0/1546919371 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6877 (secure 0 0 0) 0x7f00bc09b900 con 0x7f00c81017d0 2024-03-21T08:54:51.213 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:51.213 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8}],"filesystems":[]} 2024-03-21T08:54:51.213 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 9 2024-03-21T08:54:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f00b40a2f50 msgr2=0x7f00b40a5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f00b40a2f50 0x7f00b40a5410 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f00c4000c00 tx=0x7f00c4005fd0 comp rx=0 tx=0).stop 2024-03-21T08:54:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 msgr2=0x7f00c819eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 0x7f00c819eb70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f00bc00d770 tx=0x7f00bc00dc30 comp rx=0 tx=0).stop 2024-03-21T08:54:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 shutdown_connections 2024-03-21T08:54:51.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f00b40a2f50 0x7f00b40a5410 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f00c8102190 0x7f00c81a10c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f00c81017d0 0x7f00c819eb70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 --2- 172.21.15.87:0/1546919371 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f00c81005e0 0x7f00c806a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 >> 172.21.15.87:0/1546919371 conn(0x7f00c80fbb50 msgr2=0x7f00c8104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 shutdown_connections 2024-03-21T08:54:51.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:51.219+0000 7f00cf612640 1 -- 172.21.15.87:0/1546919371 wait complete. 2024-03-21T08:54:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:50 smithi100 ceph-mon[36306]: Deploying daemon mds.i on smithi105 2024-03-21T08:54:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:50 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6836/2948361419,v1:172.21.15.100:6837/2948361419] up:boot 2024-03-21T08:54:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:50 smithi100 ceph-mon[36306]: fsmap 8 up:standby 2024-03-21T08:54:51.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:50 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T08:54:51.926 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 08:54:51 smithi105 systemd[1]: Started Ceph mds.i for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:52.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:51 smithi105 ceph-mon[35993]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:52.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:51 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1546919371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:51 smithi100 ceph-mon[36306]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:52.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:51 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1546919371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:52.379 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:51 smithi087 ceph-mon[32385]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:52.379 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:51 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1546919371' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:53.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] up:boot 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: fsmap 9 up:standby 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:53.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:52 smithi087 ceph-mon[32385]: Deploying daemon mds.g on smithi087 2024-03-21T08:54:53.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:53.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] up:boot 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: fsmap 9 up:standby 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:53.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:52 smithi105 ceph-mon[35993]: Deploying daemon mds.g on smithi087 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] up:boot 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: fsmap 9 up:standby 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:53.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:52 smithi100 ceph-mon[36306]: Deploying daemon mds.g on smithi087 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 -- 172.21.15.87:0/643336579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 msgr2=0x7f5544106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 --2- 172.21.15.87:0/643336579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f5544106e60 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f552c009900 tx=0x7f552c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 -- 172.21.15.87:0/643336579 shutdown_connections 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 --2- 172.21.15.87:0/643336579 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55441073a0 0x7f55441097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 --2- 172.21.15.87:0/643336579 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f5544106e60 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 --2- 172.21.15.87:0/643336579 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5544102140 0x7f5544104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 -- 172.21.15.87:0/643336579 >> 172.21.15.87:0/643336579 conn(0x7f55440fbb30 msgr2=0x7f55440fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:53.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 -- 172.21.15.87:0/643336579 shutdown_connections 2024-03-21T08:54:53.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.340+0000 7f5548e27640 1 -- 172.21.15.87:0/643336579 wait complete. 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 Processor -- start 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 -- start start 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5544102140 0x7f5544199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f554419a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55441073a0 0x7f55441a1180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554410d340 con 0x7f5544104a70 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554410d1c0 con 0x7f5544102140 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f5548e27640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554410d4c0 con 0x7f55441073a0 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f553bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f554419a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f553bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f554419a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47536/0 (socket says 172.21.15.87:47536) 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.341+0000 7f553bfff640 1 -- 172.21.15.87:0/3982489722 learned_addr learned my addr 172.21.15.87:0/3982489722 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:53.339 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f554259c640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5544102140 0x7f5544199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553bfff640 1 -- 172.21.15.87:0/3982489722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55441073a0 msgr2=0x7f55441a1180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553bfff640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55441073a0 0x7f55441a1180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553bfff640 1 -- 172.21.15.87:0/3982489722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5544102140 msgr2=0x7f5544199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553bfff640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5544102140 0x7f5544199bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553bfff640 1 -- 172.21.15.87:0/3982489722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55441a17f0 con 0x7f5544104a70 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553bfff640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f554419a110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f552c005ab0 tx=0x7f552c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c041070 con 0x7f5544104a70 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55441a1a80 con 0x7f5544104a70 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f552c0054e0 con 0x7f5544104a70 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f552c034440 con 0x7f5544104a70 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.342+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55441a1f70 con 0x7f5544104a70 2024-03-21T08:54:53.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.343+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5544108ef0 con 0x7f5544104a70 2024-03-21T08:54:53.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.344+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f552c044070 con 0x7f5544104a70 2024-03-21T08:54:53.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.345+0000 7f553b7fe640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f55300a2f20 0x7f55300a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:53.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.345+0000 7f554259c640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f55300a2f20 0x7f55300a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:53.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.345+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f552c0f0e20 con 0x7f5544104a70 2024-03-21T08:54:53.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.345+0000 7f554259c640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f55300a2f20 0x7f55300a53e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55340058c0 tx=0x7f5534005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:53.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.348+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f552c0be0f0 con 0x7f5544104a70 2024-03-21T08:54:53.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.565+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f554402d070 con 0x7f5544104a70 2024-03-21T08:54:53.561 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.566+0000 7f553b7fe640 1 -- 172.21.15.87:0/3982489722 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7670 (secure 0 0 0) 0x7f552c0bd9f0 con 0x7f5544104a70 2024-03-21T08:54:53.561 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:53.561 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T08:54:53.562 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 10 2024-03-21T08:54:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f55300a2f20 msgr2=0x7f55300a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:53.563 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f55300a2f20 0x7f55300a53e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f55340058c0 tx=0x7f5534005c60 comp rx=0 tx=0).stop 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 msgr2=0x7f554419a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f554419a110 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f552c005ab0 tx=0x7f552c005dc0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 shutdown_connections 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f55300a2f20 0x7f55300a53e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f55441073a0 0x7f55441a1180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5544104a70 0x7f554419a110 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 --2- 172.21.15.87:0/3982489722 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5544102140 0x7f5544199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 >> 172.21.15.87:0/3982489722 conn(0x7f55440fbb30 msgr2=0x7f55441018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 shutdown_connections 2024-03-21T08:54:53.564 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:53.568+0000 7f5548e27640 1 -- 172.21.15.87:0/3982489722 wait complete. 2024-03-21T08:54:54.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:53 smithi087 ceph-mon[32385]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:54.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:53 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3982489722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:54.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:53 smithi105 ceph-mon[35993]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:54.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:53 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3982489722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:54.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:53 smithi100 ceph-mon[36306]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:54.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:53 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3982489722' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:54.906 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 08:54:54 smithi087 systemd[1]: Started Ceph mds.g for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:55.574 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:55.681 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 -- 172.21.15.87:0/1832217272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 msgr2=0x7fd1b4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/1832217272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4106e60 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8009900 tx=0x7fd1a802bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 -- 172.21.15.87:0/1832217272 shutdown_connections 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/1832217272 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 0x7fd1b41097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/1832217272 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4106e60 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/1832217272 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102140 0x7fd1b4104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 -- 172.21.15.87:0/1832217272 >> 172.21.15.87:0/1832217272 conn(0x7fd1b40fbb30 msgr2=0x7fd1b40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 -- 172.21.15.87:0/1832217272 shutdown_connections 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.683+0000 7fd1b9b62640 1 -- 172.21.15.87:0/1832217272 wait complete. 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 Processor -- start 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 -- start start 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102140 0x7fd1b4197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:55.682 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 0x7fd1b419efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410d390 con 0x7fd1b4104a70 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410d210 con 0x7fd1b4102140 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b9b62640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410d510 con 0x7fd1b41073a0 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47566/0 (socket says 172.21.15.87:47566) 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.684+0000 7fd1b2ffd640 1 -- 172.21.15.87:0/3451506374 learned_addr learned my addr 172.21.15.87:0/3451506374 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b37fe640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102140 0x7fd1b4197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b3fff640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 0x7fd1b419efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b2ffd640 1 -- 172.21.15.87:0/3451506374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 msgr2=0x7fd1b419efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b2ffd640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 0x7fd1b419efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b2ffd640 1 -- 172.21.15.87:0/3451506374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102140 msgr2=0x7fd1b4197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b2ffd640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102140 0x7fd1b4197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.683 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b2ffd640 1 -- 172.21.15.87:0/3451506374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b419f620 con 0x7fd1b4104a70 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b3fff640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 0x7fd1b419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b2ffd640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4197f40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8043020 con 0x7fd1b4104a70 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a8004440 con 0x7fd1b4104a70 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b419f8b0 con 0x7fd1b4104a70 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a803e450 con 0x7fd1b4104a70 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.685+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b419fe00 con 0x7fd1b4104a70 2024-03-21T08:54:55.684 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.686+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b40fd620 con 0x7fd1b4104a70 2024-03-21T08:54:55.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.688+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd1a8046020 con 0x7fd1b4104a70 2024-03-21T08:54:55.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.688+0000 7fd1b0ff9640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd19c0a2f20 0x7fd19c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:55.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.688+0000 7fd1b37fe640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd19c0a2f20 0x7fd19c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:55.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.688+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd1a80f1690 con 0x7fd1b4104a70 2024-03-21T08:54:55.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.689+0000 7fd1b37fe640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd19c0a2f20 0x7fd19c0a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd1a40058c0 tx=0x7fd1a400a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:55.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.692+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1a804d020 con 0x7fd1b4104a70 2024-03-21T08:54:55.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.910+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1b402d2c0 con 0x7fd1b4104a70 2024-03-21T08:54:55.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.911+0000 7fd1b0ff9640 1 -- 172.21.15.87:0/3451506374 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8462 (secure 0 0 0) 0x7fd1a80be960 con 0x7fd1b4104a70 2024-03-21T08:54:55.906 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:55.906 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T08:54:55.907 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 11 2024-03-21T08:54:55.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd19c0a2f20 msgr2=0x7fd19c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd19c0a2f20 0x7fd19c0a53e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd1a40058c0 tx=0x7fd1a400a4c0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 msgr2=0x7fd1b4197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4197f40 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a8004040 comp rx=0 tx=0).stop 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 shutdown_connections 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd19c0a2f20 0x7fd19c0a53e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41073a0 0x7fd1b419efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b4104a70 0x7fd1b4197f40 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 --2- 172.21.15.87:0/3451506374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102140 0x7fd1b4197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 >> 172.21.15.87:0/3451506374 conn(0x7fd1b40fbb30 msgr2=0x7fd1b4102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 shutdown_connections 2024-03-21T08:54:55.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:55.913+0000 7fd1b9b62640 1 -- 172.21.15.87:0/3451506374 wait complete. 2024-03-21T08:54:56.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: Deploying daemon mds.a on smithi087 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.87:6838/3694307942,v1:172.21.15.87:6839/3694307942] up:boot 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: fsmap 10 up:standby 2024-03-21T08:54:56.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:54:56.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: Deploying daemon mds.a on smithi087 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.87:6838/3694307942,v1:172.21.15.87:6839/3694307942] up:boot 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: fsmap 10 up:standby 2024-03-21T08:54:56.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:54:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:56.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: Deploying daemon mds.a on smithi087 2024-03-21T08:54:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.87:6838/3694307942,v1:172.21.15.87:6839/3694307942] up:boot 2024-03-21T08:54:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: fsmap 10 up:standby 2024-03-21T08:54:56.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T08:54:57.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:56 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3451506374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:57.157 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:56 smithi087 systemd[1]: Starting Ceph mds.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:54:57.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:56 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3451506374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:57.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:56 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3451506374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:57.443 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 podman[73679]: 2024-03-21 08:54:57.089036352 +0000 UTC m=+0.018243990 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:54:57.443 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 podman[73679]: 2024-03-21 08:54:57.211636654 +0000 UTC m=+0.140844288 container create d1d898c1ca914c20a983fe421eaeff9191f3fb1eeb73f3f87684ad0ccc617f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T08:54:57.443 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 podman[73679]: 2024-03-21 08:54:57.447942924 +0000 UTC m=+0.377150574 container init d1d898c1ca914c20a983fe421eaeff9191f3fb1eeb73f3f87684ad0ccc617f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T08:54:57.821 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 podman[73679]: 2024-03-21 08:54:57.450526186 +0000 UTC m=+0.379733828 container start d1d898c1ca914c20a983fe421eaeff9191f3fb1eeb73f3f87684ad0ccc617f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-03-21T08:54:57.821 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mds[73698]: -- 172.21.15.87:0/609030510 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558794c8fa40 con 0x558794c62c00 2024-03-21T08:54:57.821 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a[73694]: starting mds.a at 2024-03-21T08:54:57.821 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 bash[73679]: d1d898c1ca914c20a983fe421eaeff9191f3fb1eeb73f3f87684ad0ccc617f2f 2024-03-21T08:54:57.821 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 08:54:57 smithi087 systemd[1]: Started Ceph mds.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:54:57.919 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:54:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 -- 172.21.15.87:0/239787460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1381005e0 msgr2=0x7fc138100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 --2- 172.21.15.87:0/239787460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1381005e0 0x7fc138100a00 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fc128009900 tx=0x7fc12802bc30 comp rx=0 tx=0).stop 2024-03-21T08:54:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 -- 172.21.15.87:0/239787460 shutdown_connections 2024-03-21T08:54:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 --2- 172.21.15.87:0/239787460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc138102190 0x7fc138108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 --2- 172.21.15.87:0/239787460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1381017d0 0x7fc138101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.028 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 --2- 172.21.15.87:0/239787460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc1381005e0 0x7fc138100a00 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 -- 172.21.15.87:0/239787460 >> 172.21.15.87:0/239787460 conn(0x7fc1380fbb50 msgr2=0x7fc1380fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 -- 172.21.15.87:0/239787460 shutdown_connections 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.030+0000 7fc13c977640 1 -- 172.21.15.87:0/239787460 wait complete. 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 Processor -- start 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 -- start start 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1381005e0 0x7fc13806a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1381017d0 0x7fc13819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 0x7fc13819ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810c250 con 0x7fc138102190 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810c0d0 con 0x7fc1381005e0 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc13c977640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13810c3d0 con 0x7fc1381017d0 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc136d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 0x7fc13819ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:58.029 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc136d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 0x7fc13819ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54232/0 (socket says 172.21.15.87:54232) 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.031+0000 7fc136d9d640 1 -- 172.21.15.87:0/367607851 learned_addr learned my addr 172.21.15.87:0/367607851 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc13659c640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1381005e0 0x7fc13806a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc135d9b640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1381017d0 0x7fc13819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc136d9d640 1 -- 172.21.15.87:0/367607851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1381017d0 msgr2=0x7fc13819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc136d9d640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1381017d0 0x7fc13819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc136d9d640 1 -- 172.21.15.87:0/367607851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1381005e0 msgr2=0x7fc13806a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc136d9d640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1381005e0 0x7fc13806a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc136d9d640 1 -- 172.21.15.87:0/367607851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc13819f600 con 0x7fc138102190 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc136d9d640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 0x7fc13819ed40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc13000eaa0 tx=0x7fc13000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc13000cdb0 con 0x7fc138102190 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc130005020 con 0x7fc138102190 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1300116b0 con 0x7fc138102190 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13819f8f0 con 0x7fc138102190 2024-03-21T08:54:58.030 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.032+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13819fe90 con 0x7fc138102190 2024-03-21T08:54:58.031 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.033+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc138107900 con 0x7fc138102190 2024-03-21T08:54:58.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.034+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc130005560 con 0x7fc138102190 2024-03-21T08:54:58.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.035+0000 7fc1277fe640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1200a2fa0 0x7fc1200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:54:58.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.035+0000 7fc13659c640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1200a2fa0 0x7fc1200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:54:58.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.035+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc1300cf0f0 con 0x7fc138102190 2024-03-21T08:54:58.032 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.035+0000 7fc13659c640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1200a2fa0 0x7fc1200a5460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc128000c00 tx=0x7fc128005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:54:58.034 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.039+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc130016030 con 0x7fc138102190 2024-03-21T08:54:58.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:58.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:58.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:58.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:58.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:58.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:58.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.252+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc13802d220 con 0x7fc138102190 2024-03-21T08:54:58.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.253+0000 7fc1277fe640 1 -- 172.21.15.87:0/367607851 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7fc13009c3c0 con 0x7fc138102190 2024-03-21T08:54:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:54:58.248 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T08:54:58.248 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 12 2024-03-21T08:54:58.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1200a2fa0 msgr2=0x7fc1200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:58.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1200a2fa0 0x7fc1200a5460 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc128000c00 tx=0x7fc128005fd0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 msgr2=0x7fc13819ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:54:58.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 0x7fc13819ed40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc13000eaa0 tx=0x7fc13000ef60 comp rx=0 tx=0).stop 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 shutdown_connections 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc1200a2fa0 0x7fc1200a5460 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc138102190 0x7fc13819ed40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1381017d0 0x7fc13819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 --2- 172.21.15.87:0/367607851 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1381005e0 0x7fc13806a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 >> 172.21.15.87:0/367607851 conn(0x7fc1380fbb50 msgr2=0x7fc138104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 shutdown_connections 2024-03-21T08:54:58.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:54:58.255+0000 7fc13c977640 1 -- 172.21.15.87:0/367607851 wait complete. 2024-03-21T08:54:58.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:54:58.256 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:54:58.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-03-21T08:54:58.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-03-21T08:54:58.257 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:54:59.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:58 smithi087 ceph-mon[32385]: Deploying daemon mds.b on smithi100 2024-03-21T08:54:59.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:58 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.87:6840/2007269086,v1:172.21.15.87:6841/2007269086] up:boot 2024-03-21T08:54:59.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:58 smithi087 ceph-mon[32385]: fsmap 11 up:standby 2024-03-21T08:54:59.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:58 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:54:59.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:58 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/367607851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:59.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:58 smithi105 ceph-mon[35993]: Deploying daemon mds.b on smithi100 2024-03-21T08:54:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:58 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.87:6840/2007269086,v1:172.21.15.87:6841/2007269086] up:boot 2024-03-21T08:54:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:58 smithi105 ceph-mon[35993]: fsmap 11 up:standby 2024-03-21T08:54:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:58 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:54:59.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:58 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/367607851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:54:59.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:58 smithi100 ceph-mon[36306]: Deploying daemon mds.b on smithi100 2024-03-21T08:54:59.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:58 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.87:6840/2007269086,v1:172.21.15.87:6841/2007269086] up:boot 2024-03-21T08:54:59.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:58 smithi100 ceph-mon[36306]: fsmap 11 up:standby 2024-03-21T08:54:59.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:58 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T08:54:59.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:58 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/367607851' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:00.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:54:59 smithi100 ceph-mon[36306]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:00.017 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:54:59 smithi100 systemd[1]: Starting Ceph mds.b for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T08:55:00.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:54:59 smithi087 ceph-mon[32385]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:00.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:54:59 smithi105 ceph-mon[35993]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:00.261 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:55:00.337 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 podman[66182]: 2024-03-21 08:54:59.978772343 +0000 UTC m=+0.017454281 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T08:55:00.337 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 podman[66182]: 2024-03-21 08:55:00.088722372 +0000 UTC m=+0.127404310 container create 32d8de4201f7ab16311c3d02a6524ad95712da9a60fa7bbb65e58d4ddae965f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T08:55:00.337 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 podman[66182]: 2024-03-21 08:55:00.341804236 +0000 UTC m=+0.380486196 container init 32d8de4201f7ab16311c3d02a6524ad95712da9a60fa7bbb65e58d4ddae965f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T08:55:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- 172.21.15.87:0/784127514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c1021f0 msgr2=0x7faf7c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- 172.21.15.87:0/784127514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c1021f0 0x7faf7c108a80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7faf74009ca0 tx=0x7faf7402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- 172.21.15.87:0/784127514 shutdown_connections 2024-03-21T08:55:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- 172.21.15.87:0/784127514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c1021f0 0x7faf7c108a80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- 172.21.15.87:0/784127514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c101830 0x7faf7c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- 172.21.15.87:0/784127514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf7c100640 0x7faf7c100a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- 172.21.15.87:0/784127514 >> 172.21.15.87:0/784127514 conn(0x7faf7c0fbbd0 msgr2=0x7faf7c0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- 172.21.15.87:0/784127514 shutdown_connections 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- 172.21.15.87:0/784127514 wait complete. 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 Processor -- start 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- start start 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf7c100640 0x7faf7c10afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 0x7faf7c109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:00.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c1021f0 0x7faf7c109b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c10d370 con 0x7faf7c101830 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c10d1f0 con 0x7faf7c100640 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.380+0000 7faf842b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf7c10d4f0 con 0x7faf7c1021f0 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 0x7faf7c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 0x7faf7c109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54254/0 (socket says 172.21.15.87:54254) 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 -- 172.21.15.87:0/2843973797 learned_addr learned my addr 172.21.15.87:0/2843973797 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8284f640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c1021f0 0x7faf7c109b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8204e640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf7c100640 0x7faf7c10afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 -- 172.21.15.87:0/2843973797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c1021f0 msgr2=0x7faf7c109b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c1021f0 0x7faf7c109b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 -- 172.21.15.87:0/2843973797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf7c100640 msgr2=0x7faf7c10afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf7c100640 0x7faf7c10afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 -- 172.21.15.87:0/2843973797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c10a3d0 con 0x7faf7c101830 2024-03-21T08:55:00.379 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8284f640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c1021f0 0x7faf7c109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:55:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf8184d640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 0x7faf7c109600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faf7800e9f0 tx=0x7faf7800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7800cd80 con 0x7faf7c101830 2024-03-21T08:55:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7faf78005500 con 0x7faf7c101830 2024-03-21T08:55:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf78010690 con 0x7faf7c101830 2024-03-21T08:55:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf7c1a19c0 con 0x7faf7c101830 2024-03-21T08:55:00.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.381+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf7c1a1ec0 con 0x7faf7c101830 2024-03-21T08:55:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.382+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf7c107890 con 0x7faf7c101830 2024-03-21T08:55:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.383+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7faf780046c0 con 0x7faf7c101830 2024-03-21T08:55:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.384+0000 7faf6f7fe640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7faf680a2f20 0x7faf680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.384+0000 7faf8204e640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7faf680a2f20 0x7faf680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.384+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7faf780cf4b0 con 0x7faf7c101830 2024-03-21T08:55:00.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.384+0000 7faf8204e640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7faf680a2f20 0x7faf680a53e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf700058c0 tx=0x7faf70005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:00.384 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.389+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7faf7809c780 con 0x7faf7c101830 2024-03-21T08:55:00.602 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.607+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faf7c02d070 con 0x7faf7c101830 2024-03-21T08:55:00.603 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.608+0000 7faf6f7fe640 1 -- 172.21.15.87:0/2843973797 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+9254 (secure 0 0 0) 0x7faf7809c080 con 0x7faf7c101830 2024-03-21T08:55:00.603 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:00.603 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T08:55:00.604 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 12 2024-03-21T08:55:00.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7faf680a2f20 msgr2=0x7faf680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:00.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7faf680a2f20 0x7faf680a53e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf700058c0 tx=0x7faf70005850 comp rx=0 tx=0).stop 2024-03-21T08:55:00.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 msgr2=0x7faf7c109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:00.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 0x7faf7c109600 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faf7800e9f0 tx=0x7faf7800edf0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.605 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 shutdown_connections 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7faf680a2f20 0x7faf680a53e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7faf7c1021f0 0x7faf7c109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7faf7c101830 0x7faf7c109600 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 --2- 172.21.15.87:0/2843973797 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7faf7c100640 0x7faf7c10afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 >> 172.21.15.87:0/2843973797 conn(0x7faf7c0fbbd0 msgr2=0x7faf7c104a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 shutdown_connections 2024-03-21T08:55:00.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:00.610+0000 7faf842b2640 1 -- 172.21.15.87:0/2843973797 wait complete. 2024-03-21T08:55:00.767 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 podman[66182]: 2024-03-21 08:55:00.344470289 +0000 UTC m=+0.383152233 container start 32d8de4201f7ab16311c3d02a6524ad95712da9a60fa7bbb65e58d4ddae965f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T08:55:00.767 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 ceph-mds[66203]: -- 172.21.15.100:0/369093522 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561f257b7a40 con 0x561f2578b000 2024-03-21T08:55:00.767 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b[66199]: starting mds.b at 2024-03-21T08:55:00.767 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 bash[66182]: 32d8de4201f7ab16311c3d02a6524ad95712da9a60fa7bbb65e58d4ddae965f4 2024-03-21T08:55:00.767 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 08:55:00 smithi100 systemd[1]: Started Ceph mds.b for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T08:55:01.724 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.724 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.724 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.724 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2843973797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] up:boot 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: fsmap 12 up:standby 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:55:01.725 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:01 smithi100 ceph-mon[36306]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:01.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2843973797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] up:boot 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: fsmap 12 up:standby 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:55:01.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:01 smithi105 ceph-mon[35993]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:01.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2843973797' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] up:boot 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: fsmap 12 up:standby 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T08:55:01.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:01 smithi087 ceph-mon[32385]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:02.617 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 -- 172.21.15.87:0/2177751707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 msgr2=0x7f9184101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 --2- 172.21.15.87:0/2177751707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f9184101c30 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9180009900 tx=0x7f918002bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 -- 172.21.15.87:0/2177751707 shutdown_connections 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 --2- 172.21.15.87:0/2177751707 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9184102170 0x7f9184108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 --2- 172.21.15.87:0/2177751707 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f9184101c30 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 --2- 172.21.15.87:0/2177751707 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f91841005c0 0x7f91841009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 -- 172.21.15.87:0/2177751707 >> 172.21.15.87:0/2177751707 conn(0x7f91840fbb30 msgr2=0x7f91840fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 -- 172.21.15.87:0/2177751707 shutdown_connections 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 -- 172.21.15.87:0/2177751707 wait complete. 2024-03-21T08:55:02.726 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 Processor -- start 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 -- start start 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 0x7f9184069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.728+0000 7f918b55b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f918419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f918b55b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184102170 0x7f918419ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f918b55b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918410c220 con 0x7f91841017b0 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f918b55b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918410c0a0 con 0x7f9184102170 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f918b55b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f918410c3a0 con 0x7f91841005c0 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f9188af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f918419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 0x7f9184069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f9188af6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f918419c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54270/0 (socket says 172.21.15.87:54270) 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f9188af6640 1 -- 172.21.15.87:0/3250309238 learned_addr learned my addr 172.21.15.87:0/3250309238 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 0x7f9184069370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:44356/0 (socket says 172.21.15.87:44356) 2024-03-21T08:55:02.727 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f9189af8640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184102170 0x7f918419ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 -- 172.21.15.87:0/3250309238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184102170 msgr2=0x7f918419ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184102170 0x7f918419ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 -- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 msgr2=0x7f918419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f918419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 -- 172.21.15.87:0/3250309238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f918419f640 con 0x7f91841005c0 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f9188af6640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f918419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f91892f7640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 0x7f9184069370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f917400e9c0 tx=0x7f917400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400ccc0 con 0x7f91841005c0 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f918419f8d0 con 0x7f91841005c0 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f91740035f0 con 0x7f91841005c0 2024-03-21T08:55:02.728 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f917400ce60 con 0x7f91841005c0 2024-03-21T08:55:02.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.729+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918419fe40 con 0x7f91841005c0 2024-03-21T08:55:02.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.730+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91841078e0 con 0x7f91841005c0 2024-03-21T08:55:02.729 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.731+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f91740037a0 con 0x7f91841005c0 2024-03-21T08:55:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.732+0000 7f917a7fc640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91700a2f20 0x7f91700a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.732+0000 7f9188af6640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91700a2f20 0x7f91700a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.732+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f91740cf2b0 con 0x7f91841005c0 2024-03-21T08:55:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.732+0000 7f9188af6640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91700a2f20 0x7f91700a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f918002c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:02.730 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.735+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f917409c580 con 0x7f91841005c0 2024-03-21T08:55:02.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.955+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f918402d2e0 con 0x7f91841005c0 2024-03-21T08:55:02.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.956+0000 7f917a7fc640 1 -- 172.21.15.87:0/3250309238 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+10046 (secure 0 0 0) 0x7f917409be80 con 0x7f91841005c0 2024-03-21T08:55:02.952 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:02.952 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5},{"gid":14634,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":11},{"gid":14646,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":24395,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":24403,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":24425,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":6},{"gid":24451,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":7},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":9},{"gid":24469,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":8},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":24487,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":10}],"filesystems":[]} 2024-03-21T08:55:02.952 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 13 2024-03-21T08:55:02.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.958+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91700a2f20 msgr2=0x7f91700a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:02.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.958+0000 7f918b55b640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91700a2f20 0x7f91700a53e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f9180000c00 tx=0x7f918002c880 comp rx=0 tx=0).stop 2024-03-21T08:55:02.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.958+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 msgr2=0x7f9184069370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:02.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.958+0000 7f918b55b640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 0x7f9184069370 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f917400e9c0 tx=0x7f917400ee80 comp rx=0 tx=0).stop 2024-03-21T08:55:02.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 shutdown_connections 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91700a2f20 0x7f91700a53e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9184102170 0x7f918419ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f91841017b0 0x7f918419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 --2- 172.21.15.87:0/3250309238 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f91841005c0 0x7f9184069370 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 >> 172.21.15.87:0/3250309238 conn(0x7f91840fbb30 msgr2=0x7f9184104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 shutdown_connections 2024-03-21T08:55:02.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:02.959+0000 7f918b55b640 1 -- 172.21.15.87:0/3250309238 wait complete. 2024-03-21T08:55:02.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:55:03.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 -- 172.21.15.87:0/2770396854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 msgr2=0x7fc104106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 --2- 172.21.15.87:0/2770396854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104106ec0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4009900 tx=0x7fc0f402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:03.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 -- 172.21.15.87:0/2770396854 shutdown_connections 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 --2- 172.21.15.87:0/2770396854 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc104107400 0x7fc1041097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 --2- 172.21.15.87:0/2770396854 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104106ec0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 --2- 172.21.15.87:0/2770396854 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc1041021a0 0x7fc104104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.088+0000 7fc10c085640 1 -- 172.21.15.87:0/2770396854 >> 172.21.15.87:0/2770396854 conn(0x7fc1040fbb30 msgr2=0x7fc1040fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 -- 172.21.15.87:0/2770396854 shutdown_connections 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 -- 172.21.15.87:0/2770396854 wait complete. 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 Processor -- start 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 -- start start 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1041021a0 0x7fc104101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc104107400 0x7fc104100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10410d340 con 0x7fc104104ad0 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10410d1c0 con 0x7fc104107400 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10c085640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc10410d4c0 con 0x7fc1041021a0 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc109620640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc109620640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104100150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54284/0 (socket says 172.21.15.87:54284) 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc109620640 1 -- 172.21.15.87:0/398442339 learned_addr learned my addr 172.21.15.87:0/398442339 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc10a622640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc104107400 0x7fc104100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.089+0000 7fc109e21640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1041021a0 0x7fc104101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc109620640 1 -- 172.21.15.87:0/398442339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1041021a0 msgr2=0x7fc104101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc109620640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1041021a0 0x7fc104101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc109620640 1 -- 172.21.15.87:0/398442339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc104107400 msgr2=0x7fc104100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc109620640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc104107400 0x7fc104100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc109620640 1 -- 172.21.15.87:0/398442339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104100f20 con 0x7fc104104ad0 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc10a622640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc104107400 0x7fc104100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:03.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc109620640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104100150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4000c00 tx=0x7fc0f402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:03.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4041070 con 0x7fc104104ad0 2024-03-21T08:55:03.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1041a1ad0 con 0x7fc104104ad0 2024-03-21T08:55:03.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc0f4004450 con 0x7fc104104ad0 2024-03-21T08:55:03.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0f4034440 con 0x7fc104104ad0 2024-03-21T08:55:03.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.090+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1041a2070 con 0x7fc104104ad0 2024-03-21T08:55:03.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.091+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1040fd620 con 0x7fc104104ad0 2024-03-21T08:55:03.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.092+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc0f4044070 con 0x7fc104104ad0 2024-03-21T08:55:03.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.093+0000 7fc0faffd640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc0f00a2f20 0x7fc0f00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.093+0000 7fc109e21640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc0f00a2f20 0x7fc0f00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.093+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc0f40f0b50 con 0x7fc104104ad0 2024-03-21T08:55:03.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.093+0000 7fc109e21640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc0f00a2f20 0x7fc0f00a53e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc100009bd0 tx=0x7fc100005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:03.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.097+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc0f40bd860 con 0x7fc104104ad0 2024-03-21T08:55:03.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.281+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc10402d270 con 0x7fc104104ad0 2024-03-21T08:55:03.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.282+0000 7fc0faffd640 1 -- 172.21.15.87:0/398442339 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+18466 (secure 0 0 0) 0x7fc0f40c1900 con 0x7fc104104ad0 2024-03-21T08:55:03.277 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:03.278 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":82,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:54:17.710738+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:55:03.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc0f00a2f20 msgr2=0x7fc0f00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc0f00a2f20 0x7fc0f00a53e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc100009bd0 tx=0x7fc100005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 msgr2=0x7fc104100150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104100150 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc0f4000c00 tx=0x7fc0f402c640 comp rx=0 tx=0).stop 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 shutdown_connections 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc0f00a2f20 0x7fc0f00a53e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc104107400 0x7fc104100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc104104ad0 0x7fc104100150 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 --2- 172.21.15.87:0/398442339 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc1041021a0 0x7fc104101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 >> 172.21.15.87:0/398442339 conn(0x7fc1040fbb30 msgr2=0x7fc104102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 shutdown_connections 2024-03-21T08:55:03.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.284+0000 7fc10c085640 1 -- 172.21.15.87:0/398442339 wait complete. 2024-03-21T08:55:03.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:55:03.291 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:55:03.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 -- 172.21.15.87:0/1907473941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c102160 msgr2=0x7f054c104550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 --2- 172.21.15.87:0/1907473941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c102160 0x7f054c104550 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f053c009900 tx=0x7f053c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 -- 172.21.15.87:0/1907473941 shutdown_connections 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 --2- 172.21.15.87:0/1907473941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c1097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 --2- 172.21.15.87:0/1907473941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c104a90 0x7f054c106e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 --2- 172.21.15.87:0/1907473941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c102160 0x7f054c104550 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 -- 172.21.15.87:0/1907473941 >> 172.21.15.87:0/1907473941 conn(0x7f054c0fbb30 msgr2=0x7f054c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 -- 172.21.15.87:0/1907473941 shutdown_connections 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.406+0000 7f055431b640 1 -- 172.21.15.87:0/1907473941 wait complete. 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 Processor -- start 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 -- start start 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c102160 0x7f054c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c104a90 0x7f054c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c10d320 con 0x7f054c1073c0 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c10d1a0 con 0x7f054c102160 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f055431b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f054c10d4a0 con 0x7f054c104a90 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54308/0 (socket says 172.21.15.87:54308) 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05518b6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c104a90 0x7f054c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 -- 172.21.15.87:0/2173961011 learned_addr learned my addr 172.21.15.87:0/2173961011 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05520b7640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c102160 0x7f054c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 -- 172.21.15.87:0/2173961011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c104a90 msgr2=0x7f054c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c104a90 0x7f054c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 -- 172.21.15.87:0/2173961011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c102160 msgr2=0x7f054c197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c102160 0x7f054c197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 -- 172.21.15.87:0/2173961011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054c19f640 con 0x7f054c1073c0 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05518b6640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c104a90 0x7f054c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05520b7640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c102160 0x7f054c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05528b8640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c19efd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f054400eb10 tx=0x7f054400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.407+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544004b20 con 0x7f054c1073c0 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.408+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f054c19f930 con 0x7f054c1073c0 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.408+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0544004cc0 con 0x7f054c1073c0 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.408+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0544010760 con 0x7f054c1073c0 2024-03-21T08:55:03.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.408+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f054c19fe80 con 0x7f054c1073c0 2024-03-21T08:55:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.409+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054c0fd620 con 0x7f054c1073c0 2024-03-21T08:55:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.410+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0544005020 con 0x7f054c1073c0 2024-03-21T08:55:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.411+0000 7f05437fe640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f05380a2f20 0x7f05380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.411+0000 7f05520b7640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f05380a2f20 0x7f05380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.411+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f05440cf4a0 con 0x7f054c1073c0 2024-03-21T08:55:03.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.411+0000 7f05520b7640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f05380a2f20 0x7f05380a53e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f053c02c7b0 tx=0x7f053c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:03.410 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.415+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f054409c770 con 0x7f054c1073c0 2024-03-21T08:55:03.574 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.574 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3250309238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.575 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/398442339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.592 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.597+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f054c0622a0 con 0x7f054c1073c0 2024-03-21T08:55:03.593 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.598+0000 7f05437fe640 1 -- 172.21.15.87:0/2173961011 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v82) v1 ==== 93+0+10 (secure 0 0 0) 0x7f054409c070 con 0x7f054c1073c0 2024-03-21T08:55:03.593 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f05380a2f20 msgr2=0x7f05380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f05380a2f20 0x7f05380a53e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f053c02c7b0 tx=0x7f053c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 msgr2=0x7f054c19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c19efd0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f054400eb10 tx=0x7f054400c2d0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 shutdown_connections 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f05380a2f20 0x7f05380a53e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f054c1073c0 0x7f054c19efd0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f054c104a90 0x7f054c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 --2- 172.21.15.87:0/2173961011 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f054c102160 0x7f054c197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 >> 172.21.15.87:0/2173961011 conn(0x7f054c0fbb30 msgr2=0x7f054c102c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:03.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 shutdown_connections 2024-03-21T08:55:03.597 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.600+0000 7f055431b640 1 -- 172.21.15.87:0/2173961011 wait complete. 2024-03-21T08:55:03.601 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.601 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3250309238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.602 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/398442339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.607 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-03-21T08:55:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 -- 172.21.15.87:0/2641862370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 msgr2=0x7f398c109810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 --2- 172.21.15.87:0/2641862370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c109810 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3984009ca0 tx=0x7f398402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 -- 172.21.15.87:0/2641862370 shutdown_connections 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 --2- 172.21.15.87:0/2641862370 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c109810 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 --2- 172.21.15.87:0/2641862370 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c104af0 0x7f398c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 --2- 172.21.15.87:0/2641862370 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c1021c0 0x7f398c1045b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.712+0000 7f3992aa1640 1 -- 172.21.15.87:0/2641862370 >> 172.21.15.87:0/2641862370 conn(0x7f398c0fbb30 msgr2=0x7f398c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 -- 172.21.15.87:0/2641862370 shutdown_connections 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 -- 172.21.15.87:0/2641862370 wait complete. 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 Processor -- start 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 -- start start 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c1021c0 0x7f398c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c104af0 0x7f398c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c10d360 con 0x7f398c107420 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c10d1e0 con 0x7f398c1021c0 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f3992aa1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c10d4e0 con 0x7f398c104af0 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f399103e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f399103e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54326/0 (socket says 172.21.15.87:54326) 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f399103e640 1 -- 172.21.15.87:0/1648351424 learned_addr learned my addr 172.21.15.87:0/1648351424 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.713+0000 7f399083d640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c1021c0 0x7f398c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f398bfff640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c104af0 0x7f398c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399103e640 1 -- 172.21.15.87:0/1648351424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c104af0 msgr2=0x7f398c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399103e640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c104af0 0x7f398c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399103e640 1 -- 172.21.15.87:0/1648351424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c1021c0 msgr2=0x7f398c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399103e640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c1021c0 0x7f398c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399103e640 1 -- 172.21.15.87:0/1648351424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c19f5f0 con 0x7f398c107420 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399083d640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c1021c0 0x7f398c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f398bfff640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c104af0 0x7f398c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f399103e640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c19ef80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3984037040 tx=0x7f3984032f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984041020 con 0x7f398c107420 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3984034d90 con 0x7f398c107420 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c19f880 con 0x7f398c107420 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3984039040 con 0x7f398c107420 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.714+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c19fd70 con 0x7f398c107420 2024-03-21T08:55:03.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.715+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0fd620 con 0x7f398c107420 2024-03-21T08:55:03.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.716+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3984044020 con 0x7f398c107420 2024-03-21T08:55:03.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.717+0000 7f3989ffb640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f39740a2f20 0x7f39740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:03.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.717+0000 7f399083d640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f39740a2f20 0x7f39740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:03.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.717+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f39840f0ff0 con 0x7f398c107420 2024-03-21T08:55:03.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.717+0000 7f399083d640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f39740a2f20 0x7f39740a53e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f397c0058c0 tx=0x7f397c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:03.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.721+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f398404b020 con 0x7f398c107420 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:55:03.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:55:03.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:03.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:03.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3250309238' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.768 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/398442339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:03.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.915+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f398c0622a0 con 0x7f398c107420 2024-03-21T08:55:03.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.915+0000 7f3989ffb640 1 -- 172.21.15.87:0/1648351424 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) v1 ==== 84+0+65 (secure 0 0 0) 0x7f39840be2c0 con 0x7f398c107420 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout:[ 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout: "poolnum": 1, 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout: "poolname": ".mgr" 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T08:55:03.911 INFO:teuthology.orchestra.run.smithi087.stdout:] 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f39740a2f20 msgr2=0x7f39740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f39740a2f20 0x7f39740a53e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f397c0058c0 tx=0x7f397c005850 comp rx=0 tx=0).stop 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 msgr2=0x7f398c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c19ef80 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3984037040 tx=0x7f3984032f20 comp rx=0 tx=0).stop 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 shutdown_connections 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f39740a2f20 0x7f39740a53e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f398c107420 0x7f398c19ef80 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f398c104af0 0x7f398c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 --2- 172.21.15.87:0/1648351424 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f398c1021c0 0x7f398c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:03.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 >> 172.21.15.87:0/1648351424 conn(0x7f398c0fbb30 msgr2=0x7f398c102b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:03.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 shutdown_connections 2024-03-21T08:55:03.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:03.917+0000 7f3992aa1640 1 -- 172.21.15.87:0/1648351424 wait complete. 2024-03-21T08:55:03.924 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-03-21T08:55:03.924 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-03-21T08:55:04.036 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 -- 172.21.15.87:0/1591320612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a341021c0 msgr2=0x7f3a341045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 --2- 172.21.15.87:0/1591320612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a341021c0 0x7f3a341045b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3a24009900 tx=0x7f3a2402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 -- 172.21.15.87:0/1591320612 shutdown_connections 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 --2- 172.21.15.87:0/1591320612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a34107420 0x7f3a34109810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 --2- 172.21.15.87:0/1591320612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a34104af0 0x7f3a34106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 --2- 172.21.15.87:0/1591320612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a341021c0 0x7f3a341045b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 -- 172.21.15.87:0/1591320612 >> 172.21.15.87:0/1591320612 conn(0x7f3a340fbb30 msgr2=0x7f3a340fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 -- 172.21.15.87:0/1591320612 shutdown_connections 2024-03-21T08:55:04.037 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.041+0000 7f3a391ff640 1 -- 172.21.15.87:0/1591320612 wait complete. 2024-03-21T08:55:04.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 Processor -- start 2024-03-21T08:55:04.039 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 -- start start 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a341021c0 0x7f3a34197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 0x7f3a34197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a34107420 0x7f3a3419eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3410d340 con 0x7f3a34104af0 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3410d1c0 con 0x7f3a341021c0 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a391ff640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3410d4c0 con 0x7f3a34107420 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a3259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 0x7f3a34197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a3259c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 0x7f3a34197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54342/0 (socket says 172.21.15.87:54342) 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a3259c640 1 -- 172.21.15.87:0/932435082 learned_addr learned my addr 172.21.15.87:0/932435082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a32d9d640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a341021c0 0x7f3a34197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.042+0000 7f3a3359e640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a34107420 0x7f3a3419eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a3259c640 1 -- 172.21.15.87:0/932435082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a34107420 msgr2=0x7f3a3419eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:04.040 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a3259c640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a34107420 0x7f3a3419eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a3259c640 1 -- 172.21.15.87:0/932435082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a341021c0 msgr2=0x7f3a34197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a3259c640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a341021c0 0x7f3a34197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a3259c640 1 -- 172.21.15.87:0/932435082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3419f660 con 0x7f3a34104af0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a32d9d640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a341021c0 0x7f3a34197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a3259c640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 0x7f3a34197f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800e9c0 tx=0x7f3a2800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2800cd80 con 0x7f3a34104af0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3a28005500 con 0x7f3a34104af0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3419f950 con 0x7f3a34104af0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a28010690 con 0x7f3a34104af0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.043+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3419fea0 con 0x7f3a34104af0 2024-03-21T08:55:04.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.044+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a340fd620 con 0x7f3a34104af0 2024-03-21T08:55:04.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.045+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3a280046c0 con 0x7f3a34104af0 2024-03-21T08:55:04.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.046+0000 7f3a1bfff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a1c0a2f20 0x7f3a1c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:04.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.046+0000 7f3a32d9d640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a1c0a2f20 0x7f3a1c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:04.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.046+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3a280cf290 con 0x7f3a34104af0 2024-03-21T08:55:04.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.046+0000 7f3a32d9d640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a1c0a2f20 0x7f3a1c0a53e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402c7b0 tx=0x7f3a24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:04.045 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.050+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3a2809c560 con 0x7f3a34104af0 2024-03-21T08:55:04.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:04.248+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3a3402d1e0 con 0x7f3a34104af0 2024-03-21T08:55:04.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2173961011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1648351424' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:04 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/932435082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2173961011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1648351424' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:04 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/932435082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T08:55:04.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2173961011' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1648351424' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:04.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:04 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/932435082' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-03-21T08:55:05.208 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.211+0000 7f3a1bfff640 1 -- 172.21.15.87:0/932435082 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3a2809be60 con 0x7f3a34104af0 2024-03-21T08:55:05.208 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' created 2024-03-21T08:55:05.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a1c0a2f20 msgr2=0x7f3a1c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:05.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a1c0a2f20 0x7f3a1c0a53e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3a2402c7b0 tx=0x7f3a24005fd0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 msgr2=0x7f3a34197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 0x7f3a34197f80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3a2800e9c0 tx=0x7f3a2800ee80 comp rx=0 tx=0).stop 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 shutdown_connections 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3a1c0a2f20 0x7f3a1c0a53e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3a34107420 0x7f3a3419eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3a34104af0 0x7f3a34197f80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 --2- 172.21.15.87:0/932435082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3a341021c0 0x7f3a34197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 >> 172.21.15.87:0/932435082 conn(0x7f3a340fbb30 msgr2=0x7f3a34102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 shutdown_connections 2024-03-21T08:55:05.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.214+0000 7f3a391ff640 1 -- 172.21.15.87:0/932435082 wait complete. 2024-03-21T08:55:05.224 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-03-21T08:55:05.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 -- 172.21.15.87:0/237838462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578102120 msgr2=0x7fa578104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:05.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/237838462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578102120 0x7fa578104510 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa568009900 tx=0x7fa56802bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:05.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 -- 172.21.15.87:0/237838462 shutdown_connections 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/237838462 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa578107380 0x7fa5781097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/237838462 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa578104a50 0x7fa578106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/237838462 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578102120 0x7fa578104510 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 -- 172.21.15.87:0/237838462 >> 172.21.15.87:0/237838462 conn(0x7fa5780fbb30 msgr2=0x7fa5780fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 -- 172.21.15.87:0/237838462 shutdown_connections 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.335+0000 7fa57dfe6640 1 -- 172.21.15.87:0/237838462 wait complete. 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 Processor -- start 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 -- start start 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa578102120 0x7fa578197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa578104a50 0x7fa578197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 0x7fa57819eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57810d340 con 0x7fa578107380 2024-03-21T08:55:05.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57810d1c0 con 0x7fa578102120 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa57dfe6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57810d4c0 con 0x7fa578104a50 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa577fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 0x7fa57819eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa577fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 0x7fa57819eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54358/0 (socket says 172.21.15.87:54358) 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.336+0000 7fa577fff640 1 -- 172.21.15.87:0/2429677295 learned_addr learned my addr 172.21.15.87:0/2429677295 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa5777fe640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa578102120 0x7fa578197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa56effd640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa578104a50 0x7fa578197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa577fff640 1 -- 172.21.15.87:0/2429677295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa578104a50 msgr2=0x7fa578197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa577fff640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa578104a50 0x7fa578197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa577fff640 1 -- 172.21.15.87:0/2429677295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa578102120 msgr2=0x7fa578197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa577fff640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa578102120 0x7fa578197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa577fff640 1 -- 172.21.15.87:0/2429677295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57819f660 con 0x7fa578107380 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa577fff640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 0x7fa57819eff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa57000eaa0 tx=0x7fa57000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa57000cdb0 con 0x7fa578107380 2024-03-21T08:55:05.336 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa57819f950 con 0x7fa578107380 2024-03-21T08:55:05.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa570005020 con 0x7fa578107380 2024-03-21T08:55:05.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5700116d0 con 0x7fa578107380 2024-03-21T08:55:05.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.337+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57819fef0 con 0x7fa578107380 2024-03-21T08:55:05.337 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.338+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5780fd620 con 0x7fa578107380 2024-03-21T08:55:05.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.339+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa5700054d0 con 0x7fa578107380 2024-03-21T08:55:05.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.340+0000 7fa5757fa640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa5640a2f20 0x7fa5640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:05.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.340+0000 7fa5777fe640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa5640a2f20 0x7fa5640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:05.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.340+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7497+0+0 (secure 0 0 0) 0x7fa5700cf370 con 0x7fa578107380 2024-03-21T08:55:05.338 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.340+0000 7fa5777fe640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa5640a2f20 0x7fa5640a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa568006010 tx=0x7fa568000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:05.340 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.344+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa570016030 con 0x7fa578107380 2024-03-21T08:55:05.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:05.522+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa57802d070 con 0x7fa578107380 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/932435082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T08:55:05.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:05 smithi100 ceph-mon[36306]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T08:55:05.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/932435082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T08:55:05.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:05 smithi087 ceph-mon[32385]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/932435082' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-03-21T08:55:05.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:05 smithi105 ceph-mon[35993]: osdmap e83: 12 total, 12 up, 12 in 2024-03-21T08:55:06.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.214+0000 7fa5757fa640 1 -- 172.21.15.87:0/2429677295 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa57009c4d0 con 0x7fa578107380 2024-03-21T08:55:06.210 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' created 2024-03-21T08:55:06.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.216+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa5640a2f20 msgr2=0x7fa5640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:06.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa5640a2f20 0x7fa5640a53e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fa568006010 tx=0x7fa568000bc0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 msgr2=0x7fa57819eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:06.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 0x7fa57819eff0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fa57000eaa0 tx=0x7fa57000ef60 comp rx=0 tx=0).stop 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 shutdown_connections 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa5640a2f20 0x7fa5640a53e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa578107380 0x7fa57819eff0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa578104a50 0x7fa578197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 --2- 172.21.15.87:0/2429677295 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa578102120 0x7fa578197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 >> 172.21.15.87:0/2429677295 conn(0x7fa5780fbb30 msgr2=0x7fa578102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 shutdown_connections 2024-03-21T08:55:06.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.217+0000 7fa57dfe6640 1 -- 172.21.15.87:0/2429677295 wait complete. 2024-03-21T08:55:06.234 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-03-21T08:55:06.341 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 -- 172.21.15.87:0/613918581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 msgr2=0x7f971c101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 --2- 172.21.15.87:0/613918581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c101c30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9718009900 tx=0x7f971802bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 -- 172.21.15.87:0/613918581 shutdown_connections 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 --2- 172.21.15.87:0/613918581 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f971c102170 0x7f971c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 --2- 172.21.15.87:0/613918581 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c101c30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 --2- 172.21.15.87:0/613918581 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 0x7f971c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 -- 172.21.15.87:0/613918581 >> 172.21.15.87:0/613918581 conn(0x7f971c0fbb50 msgr2=0x7f971c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 -- 172.21.15.87:0/613918581 shutdown_connections 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 -- 172.21.15.87:0/613918581 wait complete. 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 Processor -- start 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.342+0000 7f97240b2640 1 -- start start 2024-03-21T08:55:06.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 0x7f971c06a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f971c102170 0x7f971c19ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c10c240 con 0x7f971c1017b0 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c10c0c0 con 0x7f971c1005c0 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f971c10c3c0 con 0x7f971c102170 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54384/0 (socket says 172.21.15.87:54384) 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 -- 172.21.15.87:0/3574251759 learned_addr learned my addr 172.21.15.87:0/3574251759 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f9721e4e640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 0x7f971c06a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972264f640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f971c102170 0x7f971c19ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 -- 172.21.15.87:0/3574251759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f971c102170 msgr2=0x7f971c19ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f971c102170 0x7f971c19ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 -- 172.21.15.87:0/3574251759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 msgr2=0x7f971c06a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 0x7f971c06a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 -- 172.21.15.87:0/3574251759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f971c19f610 con 0x7f971c1017b0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f9721e4e640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 0x7f971c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f972164d640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c19c710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9718009230 tx=0x7f9718005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718041070 con 0x7f971c1017b0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f97180054e0 con 0x7f971c1017b0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f971c19f8a0 con 0x7f971c1017b0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718034440 con 0x7f971c1017b0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.343+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f971c19fde0 con 0x7f971c1017b0 2024-03-21T08:55:06.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.344+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f971c1078e0 con 0x7f971c1017b0 2024-03-21T08:55:06.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.346+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9718044070 con 0x7f971c1017b0 2024-03-21T08:55:06.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.346+0000 7f9712ffd640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97040a2fa0 0x7f97040a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:06.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.346+0000 7f9721e4e640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97040a2fa0 0x7f97040a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:06.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.346+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f97180f12b0 con 0x7f971c1017b0 2024-03-21T08:55:06.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.347+0000 7f9721e4e640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97040a2fa0 0x7f97040a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f971c101610 tx=0x7f9714009540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:06.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.350+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f97180be290 con 0x7f971c1017b0 2024-03-21T08:55:06.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:06.565+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f971c062310 con 0x7f971c1017b0 2024-03-21T08:55:06.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:06 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2429677295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T08:55:06.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:06 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2429677295' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T08:55:06.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:06 smithi100 ceph-mon[36306]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T08:55:06.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:06 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2429677295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T08:55:06.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:06 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2429677295' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T08:55:06.779 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:06 smithi087 ceph-mon[32385]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T08:55:06.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:06 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2429677295' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-03-21T08:55:06.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:06 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2429677295' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-03-21T08:55:06.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:06 smithi105 ceph-mon[35993]: osdmap e84: 12 total, 12 up, 12 in 2024-03-21T08:55:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.516+0000 7f9712ffd640 1 -- 172.21.15.87:0/3574251759 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f97180bdb90 con 0x7f971c1017b0 2024-03-21T08:55:07.512 INFO:teuthology.orchestra.run.smithi087.stderr:new fs with metadata pool 2 and data pool 3 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97040a2fa0 msgr2=0x7f97040a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97040a2fa0 0x7f97040a5460 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f971c101610 tx=0x7f9714009540 comp rx=0 tx=0).stop 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 msgr2=0x7f971c19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c19c710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9718009230 tx=0x7f9718005dd0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 shutdown_connections 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f97040a2fa0 0x7f97040a5460 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f971c102170 0x7f971c19ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f971c1017b0 0x7f971c19c710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 --2- 172.21.15.87:0/3574251759 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f971c1005c0 0x7f971c06a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 >> 172.21.15.87:0/3574251759 conn(0x7f971c0fbb50 msgr2=0x7f971c104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 shutdown_connections 2024-03-21T08:55:07.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.520+0000 7f97240b2640 1 -- 172.21.15.87:0/3574251759 wait complete. 2024-03-21T08:55:07.538 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:55:07.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 -- 172.21.15.87:0/2359920966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601005e0 msgr2=0x7fec60100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 --2- 172.21.15.87:0/2359920966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601005e0 0x7fec60100a00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fec54009900 tx=0x7fec5402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:07.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 -- 172.21.15.87:0/2359920966 shutdown_connections 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 --2- 172.21.15.87:0/2359920966 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec60102190 0x7fec60108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 --2- 172.21.15.87:0/2359920966 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec601017d0 0x7fec60101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 --2- 172.21.15.87:0/2359920966 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601005e0 0x7fec60100a00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 -- 172.21.15.87:0/2359920966 >> 172.21.15.87:0/2359920966 conn(0x7fec600fbb50 msgr2=0x7fec600fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 -- 172.21.15.87:0/2359920966 shutdown_connections 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 -- 172.21.15.87:0/2359920966 wait complete. 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.649+0000 7fec66ec6640 1 Processor -- start 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 -- start start 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec601005e0 0x7fec60104570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 0x7fec60102bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60102190 0x7fec601030f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6010c250 con 0x7fec601017d0 2024-03-21T08:55:07.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6010c0d0 con 0x7fec60102190 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec6010c3d0 con 0x7fec601005e0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 0x7fec60102bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 0x7fec60102bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44682/0 (socket says 172.21.15.87:44682) 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 -- 172.21.15.87:0/624422845 learned_addr learned my addr 172.21.15.87:0/624422845 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec65463640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60102190 0x7fec601030f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec64c62640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec601005e0 0x7fec60104570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 -- 172.21.15.87:0/624422845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec601005e0 msgr2=0x7fec60104570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec601005e0 0x7fec60104570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 -- 172.21.15.87:0/624422845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60102190 msgr2=0x7fec601030f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60102190 0x7fec601030f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 -- 172.21.15.87:0/624422845 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec60103980 con 0x7fec601017d0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5ffff640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 0x7fec60102bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c920 tx=0x7fec4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c007c70 con 0x7fec601017d0 2024-03-21T08:55:07.649 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec601a1ab0 con 0x7fec601017d0 2024-03-21T08:55:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fec4c007e10 con 0x7fec601017d0 2024-03-21T08:55:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.650+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c018890 con 0x7fec601017d0 2024-03-21T08:55:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.651+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec601a2000 con 0x7fec601017d0 2024-03-21T08:55:07.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.651+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec601a2230 con 0x7fec601017d0 2024-03-21T08:55:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.652+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fec4c018a70 con 0x7fec601017d0 2024-03-21T08:55:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.653+0000 7fec5dffb640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fec500a2f20 0x7fec500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.653+0000 7fec64c62640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fec500a2f20 0x7fec500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.653+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fec4c0cf4d0 con 0x7fec601017d0 2024-03-21T08:55:07.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.653+0000 7fec64c62640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fec500a2f20 0x7fec500a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec54036040 tx=0x7fec54005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:07.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.657+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fec4c014030 con 0x7fec601017d0 2024-03-21T08:55:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:07 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3574251759' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T08:55:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:07 smithi100 ceph-mon[36306]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-21T08:55:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:07 smithi100 ceph-mon[36306]: pgmap v288: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:07.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.833+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec601a25c0 con 0x7fec601017d0 2024-03-21T08:55:07.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.835+0000 7fec5dffb640 1 -- 172.21.15.87:0/624422845 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22242 (secure 0 0 0) 0x7fec4c00de20 con 0x7fec601017d0 2024-03-21T08:55:07.831 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:07.831 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":85,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:55:06.566920+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fec500a2f20 msgr2=0x7fec500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fec500a2f20 0x7fec500a53e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fec54036040 tx=0x7fec54005fd0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 msgr2=0x7fec60102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 0x7fec60102bb0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00c920 tx=0x7fec4c00cde0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 shutdown_connections 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fec500a2f20 0x7fec500a53e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fec60102190 0x7fec601030f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fec601017d0 0x7fec60102bb0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 --2- 172.21.15.87:0/624422845 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fec601005e0 0x7fec60104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 >> 172.21.15.87:0/624422845 conn(0x7fec600fbb50 msgr2=0x7fec600fd590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 shutdown_connections 2024-03-21T08:55:07.834 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.837+0000 7fec66ec6640 1 -- 172.21.15.87:0/624422845 wait complete. 2024-03-21T08:55:07.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:07 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3574251759' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T08:55:07.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:07 smithi087 ceph-mon[32385]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-21T08:55:07.843 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:07 smithi087 ceph-mon[32385]: pgmap v288: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:07.844 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:55:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 -- 172.21.15.87:0/3342603186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 msgr2=0x7f72cc100a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 --2- 172.21.15.87:0/3342603186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc100a60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f72bc009900 tx=0x7f72bc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 -- 172.21.15.87:0/3342603186 shutdown_connections 2024-03-21T08:55:07.947 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 --2- 172.21.15.87:0/3342603186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc1021f0 0x7f72cc108a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 --2- 172.21.15.87:0/3342603186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72cc101830 0x7f72cc101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 --2- 172.21.15.87:0/3342603186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc100a60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 -- 172.21.15.87:0/3342603186 >> 172.21.15.87:0/3342603186 conn(0x7f72cc0fbbd0 msgr2=0x7f72cc0fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 -- 172.21.15.87:0/3342603186 shutdown_connections 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 -- 172.21.15.87:0/3342603186 wait complete. 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 Processor -- start 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.949+0000 7f72d2145640 1 -- start start 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72cc101830 0x7f72cc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc1021f0 0x7f72cc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc10c220 con 0x7f72cc100640 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc10c0a0 con 0x7f72cc101830 2024-03-21T08:55:07.948 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc10c3a0 con 0x7f72cc1021f0 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44698/0 (socket says 172.21.15.87:44698) 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 -- 172.21.15.87:0/1657044260 learned_addr learned my addr 172.21.15.87:0/1657044260 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cbfff640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc1021f0 0x7f72cc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 -- 172.21.15.87:0/1657044260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc1021f0 msgr2=0x7f72cc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc1021f0 0x7f72cc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 -- 172.21.15.87:0/1657044260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72cc101830 msgr2=0x7f72cc197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72cc101830 0x7f72cc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 -- 172.21.15.87:0/1657044260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc19f620 con 0x7f72cc100640 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72cb7fe640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc197a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000c00 tx=0x7f72bc004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc041070 con 0x7f72cc100640 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f72bc0054e0 con 0x7f72cc100640 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc19f8b0 con 0x7f72cc100640 2024-03-21T08:55:07.949 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc0346a0 con 0x7f72cc100640 2024-03-21T08:55:07.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.950+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc19fda0 con 0x7f72cc100640 2024-03-21T08:55:07.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.951+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc19ffa0 con 0x7f72cc100640 2024-03-21T08:55:07.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.952+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f72bc044070 con 0x7f72cc100640 2024-03-21T08:55:07.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.953+0000 7f72c8ff9640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f72b80a2f20 0x7f72b80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:07.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.953+0000 7f72caffd640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f72b80a2f20 0x7f72b80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:07.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.953+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f72bc0f10a0 con 0x7f72cc100640 2024-03-21T08:55:07.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.954+0000 7f72caffd640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f72b80a2f20 0x7f72b80a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72cc198f10 tx=0x7f72b4005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:07.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:07.958+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f72bc0be020 con 0x7f72cc100640 2024-03-21T08:55:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:07 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3574251759' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-03-21T08:55:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:07 smithi105 ceph-mon[35993]: daemon mds.i assigned to filesystem cephfs as rank 0 2024-03-21T08:55:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:07 smithi105 ceph-mon[35993]: pgmap v288: 97 pgs: 9 creating+peering, 1 active+clean, 87 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-03-21T08:55:08.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.134+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72cc1a0330 con 0x7f72cc100640 2024-03-21T08:55:08.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.135+0000 7f72c8ff9640 1 -- 172.21.15.87:0/1657044260 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22242 (secure 0 0 0) 0x7f72bc0bd920 con 0x7f72cc100640 2024-03-21T08:55:08.132 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:08.132 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":85,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:55:06.566920+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f72b80a2f20 msgr2=0x7f72b80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f72b80a2f20 0x7f72b80a53e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f72cc198f10 tx=0x7f72b4005bf0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 msgr2=0x7f72cc197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc197a00 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f72bc000c00 tx=0x7f72bc004040 comp rx=0 tx=0).stop 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 shutdown_connections 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f72b80a2f20 0x7f72b80a53e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f72cc1021f0 0x7f72cc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f72cc101830 0x7f72cc197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 --2- 172.21.15.87:0/1657044260 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f72cc100640 0x7f72cc197a00 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 >> 172.21.15.87:0/1657044260 conn(0x7f72cc0fbbd0 msgr2=0x7f72cc069cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:08.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 shutdown_connections 2024-03-21T08:55:08.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.137+0000 7f72d2145640 1 -- 172.21.15.87:0/1657044260 wait complete. 2024-03-21T08:55:08.145 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-03-21T08:55:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.252+0000 7f1345af7640 1 -- 172.21.15.87:0/1546875115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13401016d0 msgr2=0x7f1340101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.252+0000 7f1345af7640 1 --2- 172.21.15.87:0/1546875115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13401016d0 0x7f1340101af0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1330009900 tx=0x7f133002bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.252+0000 7f1345af7640 1 -- 172.21.15.87:0/1546875115 shutdown_connections 2024-03-21T08:55:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.252+0000 7f1345af7640 1 --2- 172.21.15.87:0/1546875115 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1340103280 0x7f1340109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 --2- 172.21.15.87:0/1546875115 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13401028c0 0x7f1340102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 --2- 172.21.15.87:0/1546875115 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f13401016d0 0x7f1340101af0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- 172.21.15.87:0/1546875115 >> 172.21.15.87:0/1546875115 conn(0x7f13400fcc60 msgr2=0x7f13400ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- 172.21.15.87:0/1546875115 shutdown_connections 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- 172.21.15.87:0/1546875115 wait complete. 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 Processor -- start 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- start start 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13401016d0 0x7f13401979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13401028c0 0x7f1340197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 0x7f134019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134010d340 con 0x7f1340103280 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134010d1c0 con 0x7f13401016d0 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.253+0000 7f1345af7640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f134010d4c0 con 0x7f13401028c0 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 0x7f134019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 0x7f134019ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44728/0 (socket says 172.21.15.87:44728) 2024-03-21T08:55:08.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 -- 172.21.15.87:0/1393014674 learned_addr learned my addr 172.21.15.87:0/1393014674 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133effd640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13401028c0 0x7f1340197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 -- 172.21.15.87:0/1393014674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13401028c0 msgr2=0x7f1340197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13401028c0 0x7f1340197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 -- 172.21.15.87:0/1393014674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13401016d0 msgr2=0x7f13401979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13401016d0 0x7f13401979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 -- 172.21.15.87:0/1393014674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f134019f5e0 con 0x7f1340103280 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133ffff640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 0x7f134019ef70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f133800cad0 tx=0x7f133800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133800ae20 con 0x7f1340103280 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f134019f8d0 con 0x7f1340103280 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1338005020 con 0x7f1340103280 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f133800f6f0 con 0x7f1340103280 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.254+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f134019fe20 con 0x7f1340103280 2024-03-21T08:55:08.253 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.255+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13401089f0 con 0x7f1340103280 2024-03-21T08:55:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.256+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f13380051d0 con 0x7f1340103280 2024-03-21T08:55:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.257+0000 7f133cff9640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13280a2f20 0x7f13280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.257+0000 7f133f7fe640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13280a2f20 0x7f13280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.257+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f13380cf770 con 0x7f1340103280 2024-03-21T08:55:08.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.257+0000 7f133f7fe640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13280a2f20 0x7f13280a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1330036040 tx=0x7f1330000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:08.256 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.260+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1338014030 con 0x7f1340103280 2024-03-21T08:55:08.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.476+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f134002d270 con 0x7f1340103280 2024-03-21T08:55:08.515 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.519+0000 7f133cff9640 1 -- 172.21.15.87:0/1393014674 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7f133809c6f0 con 0x7f1340103280 2024-03-21T08:55:08.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13280a2f20 msgr2=0x7f13280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13280a2f20 0x7f13280a53e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1330036040 tx=0x7f1330000bc0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 msgr2=0x7f134019ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 0x7f134019ef70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f133800cad0 tx=0x7f133800cf90 comp rx=0 tx=0).stop 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 shutdown_connections 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f13280a2f20 0x7f13280a53e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1340103280 0x7f134019ef70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f13401028c0 0x7f1340197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 --2- 172.21.15.87:0/1393014674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f13401016d0 0x7f13401979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.521+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 >> 172.21.15.87:0/1393014674 conn(0x7f13400fcc60 msgr2=0x7f1340105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.522+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 shutdown_connections 2024-03-21T08:55:08.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.522+0000 7f1345af7640 1 -- 172.21.15.87:0/1393014674 wait complete. 2024-03-21T08:55:08.530 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-03-21T08:55:08.530 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 -- 172.21.15.87:0/42771077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc107170 msgr2=0x7f77dc1095c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 --2- 172.21.15.87:0/42771077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc107170 0x7f77dc1095c0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f77d4009900 tx=0x7f77d402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 -- 172.21.15.87:0/42771077 shutdown_connections 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 --2- 172.21.15.87:0/42771077 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc107170 0x7f77dc1095c0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 --2- 172.21.15.87:0/42771077 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc104840 0x7f77dc106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 --2- 172.21.15.87:0/42771077 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc101f10 0x7f77dc104300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 -- 172.21.15.87:0/42771077 >> 172.21.15.87:0/42771077 conn(0x7f77dc0fb940 msgr2=0x7f77dc0fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 -- 172.21.15.87:0/42771077 shutdown_connections 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.638+0000 7f77db59e640 1 -- 172.21.15.87:0/42771077 wait complete. 2024-03-21T08:55:08.637 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 Processor -- start 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 -- start start 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 0x7f77dc197790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc104840 0x7f77dc197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc107170 0x7f77dc19ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc10d180 con 0x7f77dc101f10 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc10d000 con 0x7f77dc107170 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77db59e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc10d300 con 0x7f77dc104840 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77da59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 0x7f77dc197790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77da59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 0x7f77dc197790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44752/0 (socket says 172.21.15.87:44752) 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77da59c640 1 -- 172.21.15.87:0/1324494205 learned_addr learned my addr 172.21.15.87:0/1324494205 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.639+0000 7f77d9d9b640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc104840 0x7f77dc197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77dad9d640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc107170 0x7f77dc19ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77da59c640 1 -- 172.21.15.87:0/1324494205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc104840 msgr2=0x7f77dc197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.638 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77da59c640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc104840 0x7f77dc197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77da59c640 1 -- 172.21.15.87:0/1324494205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc107170 msgr2=0x7f77dc19ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77da59c640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc107170 0x7f77dc19ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77da59c640 1 -- 172.21.15.87:0/1324494205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc19f3b0 con 0x7f77dc101f10 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77da59c640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 0x7f77dc197790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00ba50 tx=0x7f77cc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc019070 con 0x7f77dc101f10 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc19f6a0 con 0x7f77dc101f10 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77cc00cbe0 con 0x7f77dc101f10 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc0076e0 con 0x7f77dc101f10 2024-03-21T08:55:08.639 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.640+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc19fc40 con 0x7f77dc101f10 2024-03-21T08:55:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.641+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc0fd430 con 0x7f77dc101f10 2024-03-21T08:55:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.642+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f77cc01c070 con 0x7f77dc101f10 2024-03-21T08:55:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.643+0000 7f77d37fe640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a2f20 0x7f77c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.643+0000 7f77d9d9b640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a2f20 0x7f77c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.643+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f77cc0ce810 con 0x7f77dc101f10 2024-03-21T08:55:08.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.643+0000 7f77d9d9b640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a2f20 0x7f77c80a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77dc198ca0 tx=0x7f77c400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:08.643 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.647+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77cc09b140 con 0x7f77dc101f10 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3574251759' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/624422845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1657044260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:08.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:08 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1393014674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T08:55:08.855 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:08.859+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) v1 -- 0x7f77dc0622a0 con 0x7f77dc101f10 2024-03-21T08:55:08.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T08:55:08.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T08:55:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3574251759' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T08:55:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-21T08:55:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-21T08:55:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/624422845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1657044260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:08 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1393014674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T08:55:08.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-21T08:55:08.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: osdmap e85: 12 total, 12 up, 12 in 2024-03-21T08:55:08.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3574251759' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-03-21T08:55:08.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: fsmap cephfs:1 {0=i=up:creating} 11 up:standby 2024-03-21T08:55:08.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: daemon mds.i is now active in filesystem cephfs as rank 0 2024-03-21T08:55:08.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/624422845' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:08.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1657044260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:08.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:08 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1393014674' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-03-21T08:55:09.531 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.535+0000 7f77d37fe640 1 -- 172.21.15.87:0/1324494205 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v16) v1 ==== 105+0+0 (secure 0 0 0) 0x7f77cc09f1e0 con 0x7f77dc101f10 2024-03-21T08:55:09.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a2f20 msgr2=0x7f77c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a2f20 0x7f77c80a53e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f77dc198ca0 tx=0x7f77c400b410 comp rx=0 tx=0).stop 2024-03-21T08:55:09.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 msgr2=0x7f77dc197790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.533 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 0x7f77dc197790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00ba50 tx=0x7f77cc00bf10 comp rx=0 tx=0).stop 2024-03-21T08:55:09.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 shutdown_connections 2024-03-21T08:55:09.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a2f20 0x7f77c80a53e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc107170 0x7f77dc19ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc104840 0x7f77dc197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.534 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 --2- 172.21.15.87:0/1324494205 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc101f10 0x7f77dc197790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 >> 172.21.15.87:0/1324494205 conn(0x7f77dc0fb940 msgr2=0x7f77dc102870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:09.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 shutdown_connections 2024-03-21T08:55:09.535 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.537+0000 7f77db59e640 1 -- 172.21.15.87:0/1324494205 wait complete. 2024-03-21T08:55:09.547 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-03-21T08:55:09.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.657+0000 7f8687659640 1 -- 172.21.15.87:0/3683438980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 msgr2=0x7f8680110060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.657+0000 7f8687659640 1 --2- 172.21.15.87:0/3683438980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f8680110060 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8678009930 tx=0x7f867802bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- 172.21.15.87:0/3683438980 shutdown_connections 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 --2- 172.21.15.87:0/3683438980 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f8680110060 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 --2- 172.21.15.87:0/3683438980 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680074100 0x7f8680074580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 --2- 172.21.15.87:0/3683438980 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86800756e0 0x7f8680073b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- 172.21.15.87:0/3683438980 >> 172.21.15.87:0/3683438980 conn(0x7f86800fcc10 msgr2=0x7f86800ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- 172.21.15.87:0/3683438980 shutdown_connections 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- 172.21.15.87:0/3683438980 wait complete. 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 Processor -- start 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- start start 2024-03-21T08:55:09.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680074100 0x7f868019df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86800756e0 0x7f868019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f86801a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680113910 con 0x7f868010dc70 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680113790 con 0x7f8680074100 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8687659640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680113a90 con 0x7f86800756e0 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.658+0000 7f8685bf6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f86801a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f86801a54c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44758/0 (socket says 172.21.15.87:44758) 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 -- 172.21.15.87:0/2055111804 learned_addr learned my addr 172.21.15.87:0/2055111804 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8684bf4640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86800756e0 0x7f868019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f86853f5640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680074100 0x7f868019df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:09.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 -- 172.21.15.87:0/2055111804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86800756e0 msgr2=0x7f868019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86800756e0 0x7f868019e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 -- 172.21.15.87:0/2055111804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680074100 msgr2=0x7f868019df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680074100 0x7f868019df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 -- 172.21.15.87:0/2055111804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86801a5b30 con 0x7f868010dc70 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8685bf6640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f86801a54c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8678009900 tx=0x7f8678005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678041070 con 0x7f868010dc70 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86801a5dc0 con 0x7f868010dc70 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f86780044d0 con 0x7f868010dc70 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678003e50 con 0x7f868010dc70 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.659+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86801a6220 con 0x7f868010dc70 2024-03-21T08:55:09.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.660+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680103d30 con 0x7f868010dc70 2024-03-21T08:55:09.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.661+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8678044070 con 0x7f868010dc70 2024-03-21T08:55:09.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.662+0000 7f86767fc640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f866c0a30c0 0x7f866c0a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:09.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.662+0000 7f86853f5640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f866c0a30c0 0x7f866c0a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:09.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.662+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f86780f0f60 con 0x7f868010dc70 2024-03-21T08:55:09.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.662+0000 7f86853f5640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f866c0a30c0 0x7f866c0a5580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f867c005fd0 tx=0x7f867c009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:09.662 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.666+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f86780bdee0 con 0x7f868010dc70 2024-03-21T08:55:09.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.879+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f868002d070 con 0x7f868010dc70 2024-03-21T08:55:09.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:09 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1393014674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T08:55:09.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:09 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] up:active 2024-03-21T08:55:09.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:09 smithi087 ceph-mon[32385]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-21T08:55:09.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:09 smithi087 ceph-mon[32385]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T08:55:09.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:09 smithi087 ceph-mon[32385]: pgmap v291: 97 pgs: 41 active+clean, 28 creating+peering, 28 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-03-21T08:55:09.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:09 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1324494205' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T08:55:09.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.930+0000 7f86767fc640 1 -- 172.21.15.87:0/2055111804 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) v1 ==== 121+0+0 (secure 0 0 0) 0x7f86780bd7e0 con 0x7f868010dc70 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f866c0a30c0 msgr2=0x7f866c0a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f866c0a30c0 0x7f866c0a5580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f867c005fd0 tx=0x7f867c009500 comp rx=0 tx=0).stop 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 msgr2=0x7f86801a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f86801a54c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8678009900 tx=0x7f8678005d50 comp rx=0 tx=0).stop 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 shutdown_connections 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f866c0a30c0 0x7f866c0a5580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f868010dc70 0x7f86801a54c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f86800756e0 0x7f868019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 --2- 172.21.15.87:0/2055111804 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8680074100 0x7f868019df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 >> 172.21.15.87:0/2055111804 conn(0x7f86800fcc10 msgr2=0x7f8680100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 shutdown_connections 2024-03-21T08:55:09.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:09.933+0000 7f8687659640 1 -- 172.21.15.87:0/2055111804 wait complete. 2024-03-21T08:55:09.944 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-03-21T08:55:09.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:09 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1393014674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T08:55:09.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:09 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] up:active 2024-03-21T08:55:09.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:09 smithi105 ceph-mon[35993]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-21T08:55:09.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:09 smithi105 ceph-mon[35993]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T08:55:09.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:09 smithi105 ceph-mon[35993]: pgmap v291: 97 pgs: 41 active+clean, 28 creating+peering, 28 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-03-21T08:55:09.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:09 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1324494205' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T08:55:10.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:09 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1393014674' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-03-21T08:55:10.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:09 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] up:active 2024-03-21T08:55:10.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:09 smithi100 ceph-mon[36306]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-21T08:55:10.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:09 smithi100 ceph-mon[36306]: osdmap e86: 12 total, 12 up, 12 in 2024-03-21T08:55:10.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:09 smithi100 ceph-mon[36306]: pgmap v291: 97 pgs: 41 active+clean, 28 creating+peering, 28 unknown; 577 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 5 op/s 2024-03-21T08:55:10.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:09 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1324494205' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-03-21T08:55:10.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.051+0000 7f6593b97640 1 -- 172.21.15.87:0/858668651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 msgr2=0x7f658c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:10.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.051+0000 7f6593b97640 1 --2- 172.21.15.87:0/858668651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c102e90 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f657c009930 tx=0x7f657c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:10.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- 172.21.15.87:0/858668651 shutdown_connections 2024-03-21T08:55:10.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 --2- 172.21.15.87:0/858668651 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f658c1033d0 0x7f658c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 --2- 172.21.15.87:0/858668651 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c102e90 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 --2- 172.21.15.87:0/858668651 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f658c0ff9a0 0x7f658c0ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- 172.21.15.87:0/858668651 >> 172.21.15.87:0/858668651 conn(0x7f658c0fb2d0 msgr2=0x7f658c0fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- 172.21.15.87:0/858668651 shutdown_connections 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- 172.21.15.87:0/858668651 wait complete. 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 Processor -- start 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- start start 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f658c0ff9a0 0x7f658c1979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f658c1033d0 0x7f658c19ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c10d350 con 0x7f658c102a10 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c10d1d0 con 0x7f658c0ff9a0 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6593b97640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f658c10d4d0 con 0x7f658c1033d0 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6591132640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:10.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6591132640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44784/0 (socket says 172.21.15.87:44784) 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6591132640 1 -- 172.21.15.87:0/2499665829 learned_addr learned my addr 172.21.15.87:0/2499665829 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6592134640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f658c1033d0 0x7f658c19ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.052+0000 7f6591933640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f658c0ff9a0 0x7f658c1979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6591132640 1 -- 172.21.15.87:0/2499665829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f658c1033d0 msgr2=0x7f658c19ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6591132640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f658c1033d0 0x7f658c19ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6591132640 1 -- 172.21.15.87:0/2499665829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f658c0ff9a0 msgr2=0x7f658c1979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6591132640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f658c0ff9a0 0x7f658c1979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6591132640 1 -- 172.21.15.87:0/2499665829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f658c19f5e0 con 0x7f658c102a10 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6591132640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c197f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f657c000c00 tx=0x7f657c02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c041070 con 0x7f658c102a10 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f658c19f870 con 0x7f658c102a10 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f657c004450 con 0x7f658c102a10 2024-03-21T08:55:10.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f657c034440 con 0x7f658c102a10 2024-03-21T08:55:10.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.053+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f658c19fdb0 con 0x7f658c102a10 2024-03-21T08:55:10.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.054+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f658c108bc0 con 0x7f658c102a10 2024-03-21T08:55:10.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.055+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f657c044070 con 0x7f658c102a10 2024-03-21T08:55:10.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.055+0000 7f6582ffd640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f65780a2f20 0x7f65780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:10.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.056+0000 7f6591933640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f65780a2f20 0x7f65780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:10.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.056+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f657c0f0d70 con 0x7f658c102a10 2024-03-21T08:55:10.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.056+0000 7f6591933640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f65780a2f20 0x7f65780a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f658c069a30 tx=0x7f6588005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:10.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.060+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f657c0bd730 con 0x7f658c102a10 2024-03-21T08:55:10.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.275+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f658c02d240 con 0x7f658c102a10 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: Cluster is now healthy 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1324494205' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-21T08:55:10.885 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2055111804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2055111804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] up:active 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 2 up:standby-replay 8 up:standby 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-21T08:55:10.886 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:10 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2499665829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T08:55:10.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-21T08:55:10.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: Cluster is now healthy 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1324494205' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2055111804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2055111804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] up:active 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 2 up:standby-replay 8 up:standby 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-21T08:55:10.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:10 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2499665829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T08:55:10.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.936+0000 7f6582ffd640 1 -- 172.21.15.87:0/2499665829 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f657c0c17d0 con 0x7f658c102a10 2024-03-21T08:55:10.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f65780a2f20 msgr2=0x7f65780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:10.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f65780a2f20 0x7f65780a53e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f658c069a30 tx=0x7f6588005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:10.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 msgr2=0x7f658c197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:10.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c197f00 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f657c000c00 tx=0x7f657c02c640 comp rx=0 tx=0).stop 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 shutdown_connections 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f65780a2f20 0x7f65780a53e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f658c1033d0 0x7f658c19ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f658c102a10 0x7f658c197f00 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 --2- 172.21.15.87:0/2499665829 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f658c0ff9a0 0x7f658c1979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 >> 172.21.15.87:0/2499665829 conn(0x7f658c0fb2d0 msgr2=0x7f658c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 shutdown_connections 2024-03-21T08:55:10.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:10.939+0000 7f6593b97640 1 -- 172.21.15.87:0/2499665829 wait complete. 2024-03-21T08:55:10.950 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-03-21T08:55:10.951 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-03-21T08:55:11.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-03-21T08:55:11.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: Cluster is now healthy 2024-03-21T08:55:11.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-03-21T08:55:11.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1324494205' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: daemon mds.b assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: fsmap cephfs:1 {0=i=up:active} 11 up:standby 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: fsmap cephfs:2 {0=i=up:active,1=b=up:creating} 10 up:standby 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: daemon mds.b is now active in filesystem cephfs as rank 1 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2055111804' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2055111804' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] up:active 2024-03-21T08:55:11.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: daemon mds.f assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-03-21T08:55:11.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: fsmap cephfs:2 {0=i=up:active,1=b=up:active} 2 up:standby-replay 8 up:standby 2024-03-21T08:55:11.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=f=up:creating} 2 up:standby-replay 7 up:standby 2024-03-21T08:55:11.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: daemon mds.f is now active in filesystem cephfs as rank 2 2024-03-21T08:55:11.019 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:10 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2499665829' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-03-21T08:55:11.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 -- 172.21.15.87:0/363368185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 msgr2=0x7f30f0108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:11.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 --2- 172.21.15.87:0/363368185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f0108a00 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f30e8009ca0 tx=0x7f30e802c0f0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 -- 172.21.15.87:0/363368185 shutdown_connections 2024-03-21T08:55:11.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 --2- 172.21.15.87:0/363368185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f0108a00 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 --2- 172.21.15.87:0/363368185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30f01017b0 0x7f30f0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 --2- 172.21.15.87:0/363368185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f01005c0 0x7f30f01009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 -- 172.21.15.87:0/363368185 >> 172.21.15.87:0/363368185 conn(0x7f30f00fbb50 msgr2=0x7f30f00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 -- 172.21.15.87:0/363368185 shutdown_connections 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.061+0000 7f30f66a1640 1 -- 172.21.15.87:0/363368185 wait complete. 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 Processor -- start 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 -- start start 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30f01005c0 0x7f30f0199bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f01017b0 0x7f30f019a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f01a1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f010c240 con 0x7f30f0102170 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f010c0c0 con 0x7f30f01017b0 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f66a1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30f010c3c0 con 0x7f30f01005c0 2024-03-21T08:55:11.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f4c3e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f01a1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f4c3e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f01a1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44794/0 (socket says 172.21.15.87:44794) 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30f4c3e640 1 -- 172.21.15.87:0/2080305007 learned_addr learned my addr 172.21.15.87:0/2080305007 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.062+0000 7f30effff640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30f01005c0 0x7f30f0199bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30ef7fe640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f01017b0 0x7f30f019a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f4c3e640 1 -- 172.21.15.87:0/2080305007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30f01005c0 msgr2=0x7f30f0199bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f4c3e640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30f01005c0 0x7f30f0199bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f4c3e640 1 -- 172.21.15.87:0/2080305007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f01017b0 msgr2=0x7f30f019a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f4c3e640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f01017b0 0x7f30f019a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f4c3e640 1 -- 172.21.15.87:0/2080305007 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f01a17d0 con 0x7f30f0102170 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f4c3e640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f01a1160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30e8037040 tx=0x7f30e8032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8041020 con 0x7f30f0102170 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30e8034cd0 con 0x7f30f0102170 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30f01a1a60 con 0x7f30f0102170 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30e8039040 con 0x7f30f0102170 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.063+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30f01a1f50 con 0x7f30f0102170 2024-03-21T08:55:11.063 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.064+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30f01a2180 con 0x7f30f0102170 2024-03-21T08:55:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.065+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f30e8044020 con 0x7f30f0102170 2024-03-21T08:55:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.065+0000 7f30ed7fa640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30d80a2f20 0x7f30d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.066+0000 7f30effff640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30d80a2f20 0x7f30d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.066+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f30e80f10d0 con 0x7f30f0102170 2024-03-21T08:55:11.064 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.066+0000 7f30effff640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30d80a2f20 0x7f30d80a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f30f0101610 tx=0x7f30e0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:11.066 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.070+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f30e804b020 con 0x7f30f0102170 2024-03-21T08:55:11.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.270+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f30f01a2350 con 0x7f30d80a2f20 2024-03-21T08:55:11.660 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:55:11.275+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:55:11.660 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:55:11.275+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:55:11.660 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:55:11.275+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:55:11.661 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:55:11.275+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:55:11.661 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:55:11.275+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:55:11.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.960+0000 7f30ed7fa640 1 -- 172.21.15.87:0/2080305007 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f30f01a2350 con 0x7f30d80a2f20 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.962+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30d80a2f20 msgr2=0x7f30d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30d80a2f20 0x7f30d80a53e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f30f0101610 tx=0x7f30e0005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 msgr2=0x7f30f01a1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f01a1160 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f30e8037040 tx=0x7f30e8032f40 comp rx=0 tx=0).stop 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 shutdown_connections 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30d80a2f20 0x7f30d80a53e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f30f0102170 0x7f30f01a1160 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f30f01017b0 0x7f30f019a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 --2- 172.21.15.87:0/2080305007 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f30f01005c0 0x7f30f0199bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 >> 172.21.15.87:0/2080305007 conn(0x7f30f00fbb50 msgr2=0x7f30f00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 shutdown_connections 2024-03-21T08:55:11.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:11.963+0000 7f30f66a1640 1 -- 172.21.15.87:0/2080305007 wait complete. 2024-03-21T08:55:11.971 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-03-21T08:55:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 -- 172.21.15.87:0/1159972718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32980ff980 msgr2=0x7f32980ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 --2- 172.21.15.87:0/1159972718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32980ff980 0x7f32980ffda0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3288009cd0 tx=0x7f328802c0f0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 -- 172.21.15.87:0/1159972718 shutdown_connections 2024-03-21T08:55:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 --2- 172.21.15.87:0/1159972718 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3298103310 0x7f3298109ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.081 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 --2- 172.21.15.87:0/1159972718 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3298102950 0x7f3298102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 --2- 172.21.15.87:0/1159972718 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f32980ff980 0x7f32980ffda0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 -- 172.21.15.87:0/1159972718 >> 172.21.15.87:0/1159972718 conn(0x7f32980fb2d0 msgr2=0x7f32980fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 -- 172.21.15.87:0/1159972718 shutdown_connections 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.082+0000 7f329dec6640 1 -- 172.21.15.87:0/1159972718 wait complete. 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 Processor -- start 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 -- start start 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32980ff980 0x7f3298197a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 0x7f3298197fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3298103310 0x7f329819f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329810d360 con 0x7f3298102950 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329810d1e0 con 0x7f32980ff980 2024-03-21T08:55:12.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f329dec6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329810d4e0 con 0x7f3298103310 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f3296ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 0x7f3298197fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f3296ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 0x7f3298197fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44840/0 (socket says 172.21.15.87:44840) 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f3296ffd640 1 -- 172.21.15.87:0/4268679064 learned_addr learned my addr 172.21.15.87:0/4268679064 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f3297fff640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3298103310 0x7f329819f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.083+0000 7f32977fe640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32980ff980 0x7f3298197a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3296ffd640 1 -- 172.21.15.87:0/4268679064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3298103310 msgr2=0x7f329819f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3296ffd640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3298103310 0x7f329819f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3296ffd640 1 -- 172.21.15.87:0/4268679064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32980ff980 msgr2=0x7f3298197a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3296ffd640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32980ff980 0x7f3298197a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3296ffd640 1 -- 172.21.15.87:0/4268679064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329819f680 con 0x7f3298102950 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3296ffd640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 0x7f3298197fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f328c00e9f0 tx=0x7f328c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c00cd80 con 0x7f3298102950 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f328c005500 con 0x7f3298102950 2024-03-21T08:55:12.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329819f970 con 0x7f3298102950 2024-03-21T08:55:12.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328c010690 con 0x7f3298102950 2024-03-21T08:55:12.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.084+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329819fec0 con 0x7f3298102950 2024-03-21T08:55:12.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.085+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3298108af0 con 0x7f3298102950 2024-03-21T08:55:12.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.087+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f328c0046c0 con 0x7f3298102950 2024-03-21T08:55:12.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.087+0000 7f3294ff9640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f32800a2fa0 0x7f32800a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.087+0000 7f32977fe640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f32800a2fa0 0x7f32800a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.087+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f328c0cf7e0 con 0x7f3298102950 2024-03-21T08:55:12.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.088+0000 7f32977fe640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f32800a2fa0 0x7f32800a5460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f328802c700 tx=0x7f3288032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:12.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.091+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f328c053740 con 0x7f3298102950 2024-03-21T08:55:12.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: pgmap v292: 97 pgs: 61 active+clean, 28 creating+peering, 8 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-03-21T08:55:12.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2499665829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T08:55:12.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] up:active 2024-03-21T08:55:12.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T08:55:12.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-21T08:55:12.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-03-21T08:55:12.235 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-mon[35993]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-21T08:55:12.235 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: 2024-03-21T08:55:11.946+0000 7f9b45b96700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-03-21T08:55:12.236 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: 2024-03-21T08:55:11.946+0000 7f9b45b96700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-03-21T08:55:12.236 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: 2024-03-21T08:55:11.946+0000 7f9b45b96700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-03-21T08:55:12.236 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 08:55:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: 2024-03-21T08:55:11.946+0000 7f9b45b96700 -1 mds.pinger is_rank_lagging: rank=3 was never sent ping request. 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: pgmap v292: 97 pgs: 61 active+clean, 28 creating+peering, 8 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2499665829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] up:active 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-03-21T08:55:12.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:11 smithi100 ceph-mon[36306]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-21T08:55:12.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.293+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f3298061ae0 con 0x7f32800a2fa0 2024-03-21T08:55:12.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.310+0000 7f3294ff9640 1 -- 172.21.15.87:0/4268679064 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f3298061ae0 con 0x7f32800a2fa0 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f32800a2fa0 msgr2=0x7f32800a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f32800a2fa0 0x7f32800a5460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f328802c700 tx=0x7f3288032590 comp rx=0 tx=0).stop 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 msgr2=0x7f3298197fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 0x7f3298197fa0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f328c00e9f0 tx=0x7f328c00edf0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 shutdown_connections 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f32800a2fa0 0x7f32800a5460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3298103310 0x7f329819f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3298102950 0x7f3298197fa0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 --2- 172.21.15.87:0/4268679064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f32980ff980 0x7f3298197a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 >> 172.21.15.87:0/4268679064 conn(0x7f32980fb2d0 msgr2=0x7f3298105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:12.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.312+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 shutdown_connections 2024-03-21T08:55:12.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.313+0000 7f329dec6640 1 -- 172.21.15.87:0/4268679064 wait complete. 2024-03-21T08:55:12.318 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: pgmap v292: 97 pgs: 61 active+clean, 28 creating+peering, 8 unknown; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s wr, 7 op/s 2024-03-21T08:55:12.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2499665829' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-03-21T08:55:12.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] up:active 2024-03-21T08:55:12.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: daemon mds.e assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-03-21T08:55:12.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: fsmap cephfs:3 {0=i=up:active,1=b=up:active,2=f=up:active} 2 up:standby-replay 7 up:standby 2024-03-21T08:55:12.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=f=up:active,3=e=up:creating} 2 up:standby-replay 6 up:standby 2024-03-21T08:55:12.319 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:11 smithi087 ceph-mon[32385]: daemon mds.e is now active in filesystem cephfs as rank 3 2024-03-21T08:55:12.320 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-03-21T08:55:12.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 -- 172.21.15.87:0/4163414958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee90103280 msgr2=0x7fee90109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 --2- 172.21.15.87:0/4163414958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee90103280 0x7fee90109b10 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fee88009900 tx=0x7fee8802bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:12.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 -- 172.21.15.87:0/4163414958 shutdown_connections 2024-03-21T08:55:12.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 --2- 172.21.15.87:0/4163414958 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee90103280 0x7fee90109b10 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 --2- 172.21.15.87:0/4163414958 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee901028c0 0x7fee90102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 --2- 172.21.15.87:0/4163414958 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee901016d0 0x7fee90101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 -- 172.21.15.87:0/4163414958 >> 172.21.15.87:0/4163414958 conn(0x7fee900fcc40 msgr2=0x7fee900ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:12.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 -- 172.21.15.87:0/4163414958 shutdown_connections 2024-03-21T08:55:12.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.429+0000 7fee95162640 1 -- 172.21.15.87:0/4163414958 wait complete. 2024-03-21T08:55:12.427 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 Processor -- start 2024-03-21T08:55:12.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 -- start start 2024-03-21T08:55:12.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 0x7fee901979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee901028c0 0x7fee90197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee90103280 0x7fee9019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.429 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9010d350 con 0x7fee901016d0 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9010d1d0 con 0x7fee90103280 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.430+0000 7fee95162640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee9010d4d0 con 0x7fee901028c0 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 0x7fee901979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 0x7fee901979c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44854/0 (socket says 172.21.15.87:44854) 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 -- 172.21.15.87:0/986880822 learned_addr learned my addr 172.21.15.87:0/986880822 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8e59c640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee901028c0 0x7fee90197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8f59e640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee90103280 0x7fee9019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 -- 172.21.15.87:0/986880822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee901028c0 msgr2=0x7fee90197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee901028c0 0x7fee90197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 -- 172.21.15.87:0/986880822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee90103280 msgr2=0x7fee9019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee90103280 0x7fee9019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 -- 172.21.15.87:0/986880822 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee9019f5e0 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee8ed9d640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 0x7fee901979c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fee8000eb70 tx=0x7fee8000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80003b70 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fee8000ee70 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee9019f8d0 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee80010830 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.431+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee9019fdd0 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.432+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee901089f0 con 0x7fee901016d0 2024-03-21T08:55:12.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.433+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fee80004020 con 0x7fee901016d0 2024-03-21T08:55:12.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.434+0000 7fee77fff640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fee780a2f20 0x7fee780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.434+0000 7fee8e59c640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fee780a2f20 0x7fee780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.434+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fee800cf910 con 0x7fee901016d0 2024-03-21T08:55:12.432 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.434+0000 7fee8e59c640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fee780a2f20 0x7fee780a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee90198ed0 tx=0x7fee84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:12.434 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.438+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fee8009c890 con 0x7fee901016d0 2024-03-21T08:55:12.640 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.644+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fee90061ae0 con 0x7fee780a2f20 2024-03-21T08:55:12.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.648+0000 7fee77fff640 1 -- 172.21.15.87:0/986880822 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fee90061ae0 con 0x7fee780a2f20 2024-03-21T08:55:12.644 INFO:teuthology.orchestra.run.smithi087.stdout:/volumes/qa/sv_0/b313f7b6-b526-403a-b959-bf42bf9bbad1 2024-03-21T08:55:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fee780a2f20 msgr2=0x7fee780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fee780a2f20 0x7fee780a53e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fee90198ed0 tx=0x7fee84005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 msgr2=0x7fee901979c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 0x7fee901979c0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fee8000eb70 tx=0x7fee8000c6a0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 shutdown_connections 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fee780a2f20 0x7fee780a53e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fee90103280 0x7fee9019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fee901028c0 0x7fee90197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 --2- 172.21.15.87:0/986880822 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fee901016d0 0x7fee901979c0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 >> 172.21.15.87:0/986880822 conn(0x7fee900fcc40 msgr2=0x7fee90105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 shutdown_connections 2024-03-21T08:55:12.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.650+0000 7fee95162640 1 -- 172.21.15.87:0/986880822 wait complete. 2024-03-21T08:55:12.657 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-03-21T08:55:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- 172.21.15.87:0/1147399501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40101790 msgr2=0x7f1d40101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1147399501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40101790 0x7f1d40101c10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c009900 tx=0x7f1d3c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:12.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- 172.21.15.87:0/1147399501 shutdown_connections 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1147399501 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d40102150 0x7f1d401089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1147399501 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40101790 0x7f1d40101c10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1147399501 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d401005a0 0x7f1d401009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- 172.21.15.87:0/1147399501 >> 172.21.15.87:0/1147399501 conn(0x7f1d400fbb30 msgr2=0x7f1d400fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- 172.21.15.87:0/1147399501 shutdown_connections 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- 172.21.15.87:0/1147399501 wait complete. 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 Processor -- start 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- start start 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d401005a0 0x7f1d401968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d40101790 0x7f1d40196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 0x7f1d4019de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4010c230 con 0x7f1d40102150 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.766+0000 7f1d472ea640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4010c0b0 con 0x7f1d401005a0 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d472ea640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d4010c3b0 con 0x7f1d40101790 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 0x7f1d4019de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 0x7f1d4019de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44884/0 (socket says 172.21.15.87:44884) 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 -- 172.21.15.87:0/1753755752 learned_addr learned my addr 172.21.15.87:0/1753755752 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d44885640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d40101790 0x7f1d40196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45086640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d401005a0 0x7f1d401968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 -- 172.21.15.87:0/1753755752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d40101790 msgr2=0x7f1d40196e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d40101790 0x7f1d40196e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 -- 172.21.15.87:0/1753755752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d401005a0 msgr2=0x7f1d401968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d401005a0 0x7f1d401968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 -- 172.21.15.87:0/1753755752 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d4019e4e0 con 0x7f1d40102150 2024-03-21T08:55:12.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d45887640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 0x7f1d4019de70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800eb10 tx=0x7f1d3800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:12.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38004b20 con 0x7f1d40102150 2024-03-21T08:55:12.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d4019e7d0 con 0x7f1d40102150 2024-03-21T08:55:12.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1d38004cc0 con 0x7f1d40102150 2024-03-21T08:55:12.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38010760 con 0x7f1d40102150 2024-03-21T08:55:12.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.767+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d4019ecf0 con 0x7f1d40102150 2024-03-21T08:55:12.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.768+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40107940 con 0x7f1d40102150 2024-03-21T08:55:12.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.769+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1d38005020 con 0x7f1d40102150 2024-03-21T08:55:12.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.770+0000 7f1d367fc640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1d2c0a2f20 0x7f1d2c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:12.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.770+0000 7f1d45086640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1d2c0a2f20 0x7f1d2c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:12.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.770+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1d380cf910 con 0x7f1d40102150 2024-03-21T08:55:12.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.770+0000 7f1d45086640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1d2c0a2f20 0x7f1d2c0a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d401015f0 tx=0x7f1d30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:12.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.774+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1d3809c890 con 0x7f1d40102150 2024-03-21T08:55:12.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:12.979+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d40061ae0 con 0x7f1d2c0a2f20 2024-03-21T08:55:12.997 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.001+0000 7f1d367fc640 1 -- 172.21.15.87:0/1753755752 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1d40061ae0 con 0x7f1d2c0a2f20 2024-03-21T08:55:12.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1d2c0a2f20 msgr2=0x7f1d2c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1d2c0a2f20 0x7f1d2c0a53e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1d401015f0 tx=0x7f1d30005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:12.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 msgr2=0x7f1d4019de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:12.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 0x7f1d4019de70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f1d3800eb10 tx=0x7f1d3800c2d0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 shutdown_connections 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1d2c0a2f20 0x7f1d2c0a53e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1d40102150 0x7f1d4019de70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1d40101790 0x7f1d40196e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 --2- 172.21.15.87:0/1753755752 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1d401005a0 0x7f1d401968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 >> 172.21.15.87:0/1753755752 conn(0x7f1d400fbb30 msgr2=0x7f1d40104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 shutdown_connections 2024-03-21T08:55:13.000 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.003+0000 7f1d472ea640 1 -- 172.21.15.87:0/1753755752 wait complete. 2024-03-21T08:55:13.010 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-03-21T08:55:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 -- 172.21.15.87:0/365324443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c1028a0 msgr2=0x7fa27c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 --2- 172.21.15.87:0/365324443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c1028a0 0x7fa27c102d20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa270009900 tx=0x7fa27002bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 -- 172.21.15.87:0/365324443 shutdown_connections 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 --2- 172.21.15.87:0/365324443 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27c103260 0x7fa27c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 --2- 172.21.15.87:0/365324443 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c1028a0 0x7fa27c102d20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 --2- 172.21.15.87:0/365324443 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 0x7fa27c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 -- 172.21.15.87:0/365324443 >> 172.21.15.87:0/365324443 conn(0x7fa27c0fcc40 msgr2=0x7fa27c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 -- 172.21.15.87:0/365324443 shutdown_connections 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.120+0000 7fa281b09640 1 -- 172.21.15.87:0/365324443 wait complete. 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 Processor -- start 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 -- start start 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 0x7fa27c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27c1028a0 0x7fa27c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.119 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 0x7fa27c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10d360 con 0x7fa27c103260 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10d1e0 con 0x7fa27c1016b0 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa281b09640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c10d4e0 con 0x7fa27c1028a0 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 0x7fa27c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 0x7fa27c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 0x7fa27c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44892/0 (socket says 172.21.15.87:44892) 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 -- 172.21.15.87:0/2856744670 learned_addr learned my addr 172.21.15.87:0/2856744670 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27affd640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27c1028a0 0x7fa27c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 -- 172.21.15.87:0/2856744670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27c1028a0 msgr2=0x7fa27c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27c1028a0 0x7fa27c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 -- 172.21.15.87:0/2856744670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 msgr2=0x7fa27c1979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 0x7fa27c1979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 -- 172.21.15.87:0/2856744670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa27c19f5f0 con 0x7fa27c103260 2024-03-21T08:55:13.120 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27b7fe640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 0x7fa27c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.121+0000 7fa27bfff640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 0x7fa27c19ef80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa27400eb10 tx=0x7fa27400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.122+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274004b20 con 0x7fa27c103260 2024-03-21T08:55:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.122+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa274004cc0 con 0x7fa27c103260 2024-03-21T08:55:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.122+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c19f8e0 con 0x7fa27c103260 2024-03-21T08:55:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.122+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa274010760 con 0x7fa27c103260 2024-03-21T08:55:13.121 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.122+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c19fdb0 con 0x7fa27c103260 2024-03-21T08:55:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.123+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c1089d0 con 0x7fa27c103260 2024-03-21T08:55:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.124+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa274005020 con 0x7fa27c103260 2024-03-21T08:55:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.124+0000 7fa278ff9640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2640a2f20 0x7fa2640a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.124+0000 7fa27b7fe640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2640a2f20 0x7fa2640a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.124+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa2740cf7f0 con 0x7fa27c103260 2024-03-21T08:55:13.122 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.125+0000 7fa27b7fe640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2640a2f20 0x7fa2640a53e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa27c102700 tx=0x7fa26c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:13.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.129+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa27409c770 con 0x7fa27c103260 2024-03-21T08:55:13.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:13.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] up:active 2024-03-21T08:55:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T08:55:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T08:55:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: Cluster is now healthy 2024-03-21T08:55:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-03-21T08:55:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-21T08:55:13.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:12 smithi105 ceph-mon[35993]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] up:active 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: Cluster is now healthy 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-03-21T08:55:13.268 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-21T08:55:13.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:12 smithi100 ceph-mon[36306]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-21T08:55:13.329 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.333+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa27c061ae0 con 0x7fa2640a2f20 2024-03-21T08:55:13.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.336+0000 7fa278ff9640 1 -- 172.21.15.87:0/2856744670 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+54 (secure 0 0 0) 0x7fa27c061ae0 con 0x7fa2640a2f20 2024-03-21T08:55:13.332 INFO:teuthology.orchestra.run.smithi087.stdout:/volumes/qa/sv_1/9460823c-257e-4c9f-be0e-ef702a13bc5b 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2640a2f20 msgr2=0x7fa2640a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2640a2f20 0x7fa2640a53e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fa27c102700 tx=0x7fa26c005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 msgr2=0x7fa27c19ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 0x7fa27c19ef80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fa27400eb10 tx=0x7fa27400c2d0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 shutdown_connections 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2640a2f20 0x7fa2640a53e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa27c103260 0x7fa27c19ef80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa27c1028a0 0x7fa27c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 --2- 172.21.15.87:0/2856744670 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa27c1016b0 0x7fa27c1979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 >> 172.21.15.87:0/2856744670 conn(0x7fa27c0fcc40 msgr2=0x7fa27c105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 shutdown_connections 2024-03-21T08:55:13.335 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.338+0000 7fa281b09640 1 -- 172.21.15.87:0/2856744670 wait complete. 2024-03-21T08:55:13.344 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:13.344 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] up:active 2024-03-21T08:55:13.344 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-03-21T08:55:13.344 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-03-21T08:55:13.345 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: Cluster is now healthy 2024-03-21T08:55:13.345 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: fsmap cephfs:4 {0=i=up:active,1=b=up:active,2=f=up:active,3=e=up:active} 2 up:standby-replay 6 up:standby 2024-03-21T08:55:13.345 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: fsmap cephfs:5 2 up:standby-replay 1 up:creating 5 up:standby 4 up:active 2024-03-21T08:55:13.345 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:12 smithi087 ceph-mon[32385]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-03-21T08:55:13.346 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- 172.21.15.87:0/795267146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641005c0 msgr2=0x7fe4641009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- 172.21.15.87:0/795267146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641005c0 0x7fe4641009e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe454009900 tx=0x7fe45402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- 172.21.15.87:0/795267146 shutdown_connections 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- 172.21.15.87:0/795267146 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe464102170 0x7fe464108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- 172.21.15.87:0/795267146 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641017b0 0x7fe464101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- 172.21.15.87:0/795267146 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641005c0 0x7fe4641009e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.449 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- 172.21.15.87:0/795267146 >> 172.21.15.87:0/795267146 conn(0x7fe4640fbb30 msgr2=0x7fe4640fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- 172.21.15.87:0/795267146 shutdown_connections 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- 172.21.15.87:0/795267146 wait complete. 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 Processor -- start 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- start start 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 0x7fe4641979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641017b0 0x7fe464197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe464102170 0x7fe46419ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46410c230 con 0x7fe4641005c0 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46410c0b0 con 0x7fe464102170 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.451+0000 7fe46bebf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe46410c3b0 con 0x7fe4641017b0 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 0x7fe4641979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.450 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 0x7fe4641979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44906/0 (socket says 172.21.15.87:44906) 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 -- 172.21.15.87:0/1086583830 learned_addr learned my addr 172.21.15.87:0/1086583830 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe46945a640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641017b0 0x7fe464197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe46a45c640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe464102170 0x7fe46419ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 -- 172.21.15.87:0/1086583830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641017b0 msgr2=0x7fe464197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641017b0 0x7fe464197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 -- 172.21.15.87:0/1086583830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe464102170 msgr2=0x7fe46419ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe464102170 0x7fe46419ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 -- 172.21.15.87:0/1086583830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe46419f5f0 con 0x7fe4641005c0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe469c5b640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 0x7fe4641979d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe454000c00 tx=0x7fe45402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454041070 con 0x7fe4641005c0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fe454004450 con 0x7fe4641005c0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe46419f880 con 0x7fe4641005c0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454034440 con 0x7fe4641005c0 2024-03-21T08:55:13.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.452+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe46419fd70 con 0x7fe4641005c0 2024-03-21T08:55:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.453+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe46419ffd0 con 0x7fe4641005c0 2024-03-21T08:55:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.454+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fe454044070 con 0x7fe4641005c0 2024-03-21T08:55:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.455+0000 7fe45affd640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4500a2f20 0x7fe4500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.455+0000 7fe46945a640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4500a2f20 0x7fe4500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.455+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe4540f0d70 con 0x7fe4641005c0 2024-03-21T08:55:13.453 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.455+0000 7fe46945a640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4500a2f20 0x7fe4500a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4600095f0 tx=0x7fe460005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:13.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.459+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fe4540bd730 con 0x7fe4641005c0 2024-03-21T08:55:13.670 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.674+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4641a01a0 con 0x7fe4641005c0 2024-03-21T08:55:13.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.675+0000 7fe45affd640 1 -- 172.21.15.87:0/1086583830 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11460 (secure 0 0 0) 0x7fe4540c17d0 con 0x7fe4641005c0 2024-03-21T08:55:13.672 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:13.672 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:55:12.955598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24451,24482,24425,24487,24403]},"id":1}]} 2024-03-21T08:55:13.672 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4500a2f20 msgr2=0x7fe4500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4500a2f20 0x7fe4500a53e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fe4600095f0 tx=0x7fe460005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 msgr2=0x7fe4641979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 0x7fe4641979d0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe454000c00 tx=0x7fe45402c640 comp rx=0 tx=0).stop 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 shutdown_connections 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fe4500a2f20 0x7fe4500a53e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fe464102170 0x7fe46419ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fe4641017b0 0x7fe464197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 --2- 172.21.15.87:0/1086583830 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fe4641005c0 0x7fe4641979d0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 >> 172.21.15.87:0/1086583830 conn(0x7fe4640fbb30 msgr2=0x7fe4640fd5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 shutdown_connections 2024-03-21T08:55:13.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.677+0000 7fe46bebf640 1 -- 172.21.15.87:0/1086583830 wait complete. 2024-03-21T08:55:13.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 -- 172.21.15.87:0/847032614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 msgr2=0x7f56ac101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 --2- 172.21.15.87:0/847032614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac101ad0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009900 tx=0x7f56a402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 -- 172.21.15.87:0/847032614 shutdown_connections 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 --2- 172.21.15.87:0/847032614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac103260 0x7f56ac109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 --2- 172.21.15.87:0/847032614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56ac1028a0 0x7f56ac102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 --2- 172.21.15.87:0/847032614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac101ad0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 -- 172.21.15.87:0/847032614 >> 172.21.15.87:0/847032614 conn(0x7f56ac0fcc80 msgr2=0x7f56ac0ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 -- 172.21.15.87:0/847032614 shutdown_connections 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 -- 172.21.15.87:0/847032614 wait complete. 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 Processor -- start 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.789+0000 7f56b47f4640 1 -- start start 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac1028a0 0x7f56ac197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56ac103260 0x7f56ac19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac10d330 con 0x7f56ac1016b0 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac10d1b0 con 0x7f56ac1028a0 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56ac10d4b0 con 0x7f56ac103260 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44914/0 (socket says 172.21.15.87:44914) 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 -- 172.21.15.87:0/1406855185 learned_addr learned my addr 172.21.15.87:0/1406855185 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b1d8f640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac1028a0 0x7f56ac197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2d91640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56ac103260 0x7f56ac19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 -- 172.21.15.87:0/1406855185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56ac103260 msgr2=0x7f56ac19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56ac103260 0x7f56ac19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 -- 172.21.15.87:0/1406855185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac1028a0 msgr2=0x7f56ac197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac1028a0 0x7f56ac197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 -- 172.21.15.87:0/1406855185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ac19f620 con 0x7f56ac1016b0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b1d8f640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac1028a0 0x7f56ac197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b2590640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac197a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f56a4000c00 tx=0x7f56a402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4041070 con 0x7f56ac1016b0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f56a4004450 con 0x7f56ac1016b0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56ac19f8b0 con 0x7f56ac1016b0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56a4034440 con 0x7f56ac1016b0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.790+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56ac19fda0 con 0x7f56ac1016b0 2024-03-21T08:55:13.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.791+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56ac108a20 con 0x7f56ac1016b0 2024-03-21T08:55:13.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.792+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 16) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f56a4044070 con 0x7f56ac1016b0 2024-03-21T08:55:13.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.793+0000 7f56a37fe640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f56980a2f20 0x7f56980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:13.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.793+0000 7f56b1d8f640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f56980a2f20 0x7f56980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:13.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.793+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f56a40f0d70 con 0x7f56ac1016b0 2024-03-21T08:55:13.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.793+0000 7f56b1d8f640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f56980a2f20 0x7f56980a53e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f56ac198f10 tx=0x7f569c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:13.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.798+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f56a40bd730 con 0x7f56ac1016b0 2024-03-21T08:55:13.966 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.970+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f56a40b9da0 con 0x7f56ac1016b0 2024-03-21T08:55:13.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.980+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56ac0622a0 con 0x7f56ac1016b0 2024-03-21T08:55:13.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.982+0000 7f56a37fe640 1 -- 172.21.15.87:0/1406855185 <== mon.0 v2:172.21.15.87:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22229 (secure 0 0 0) 0x7f56a40ae060 con 0x7f56ac1016b0 2024-03-21T08:55:13.978 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:13.978 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":86,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:55:08.517781+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:55:13.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f56980a2f20 msgr2=0x7f56980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f56980a2f20 0x7f56980a53e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f56ac198f10 tx=0x7f569c005f90 comp rx=0 tx=0).stop 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 msgr2=0x7f56ac197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac197a00 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f56a4000c00 tx=0x7f56a402c640 comp rx=0 tx=0).stop 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 shutdown_connections 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f56980a2f20 0x7f56980a53e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f56ac103260 0x7f56ac19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f56ac1028a0 0x7f56ac197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 --2- 172.21.15.87:0/1406855185 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f56ac1016b0 0x7f56ac197a00 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 >> 172.21.15.87:0/1406855185 conn(0x7f56ac0fcc80 msgr2=0x7f56ac105bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 shutdown_connections 2024-03-21T08:55:13.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:13.984+0000 7f56b47f4640 1 -- 172.21.15.87:0/1406855185 wait complete. 2024-03-21T08:55:13.992 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-03-21T08:55:13.992 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:55:14.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 -- 172.21.15.87:0/1469130078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01028d0 msgr2=0x7fd6e0102d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 --2- 172.21.15.87:0/1469130078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01028d0 0x7fd6e0102d50 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0009ca0 tx=0x7fd6d002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 -- 172.21.15.87:0/1469130078 shutdown_connections 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 --2- 172.21.15.87:0/1469130078 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6e0103290 0x7fd6e0109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 --2- 172.21.15.87:0/1469130078 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01028d0 0x7fd6e0102d50 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 --2- 172.21.15.87:0/1469130078 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01016e0 0x7fd6e0101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.096+0000 7fd6e7128640 1 -- 172.21.15.87:0/1469130078 >> 172.21.15.87:0/1469130078 conn(0x7fd6e00fcc50 msgr2=0x7fd6e00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 -- 172.21.15.87:0/1469130078 shutdown_connections 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 -- 172.21.15.87:0/1469130078 wait complete. 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 Processor -- start 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 -- start start 2024-03-21T08:55:14.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 0x7fd6e0197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01028d0 0x7fd6e0197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6e0103290 0x7fd6e019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010d360 con 0x7fd6e01016e0 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010d1e0 con 0x7fd6e0103290 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e7128640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e010d4e0 con 0x7fd6e01028d0 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e4ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 0x7fd6e0197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e4ec4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 0x7fd6e0197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44934/0 (socket says 172.21.15.87:44934) 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e4ec4640 1 -- 172.21.15.87:0/3054567111 learned_addr learned my addr 172.21.15.87:0/3054567111 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6dffff640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01028d0 0x7fd6e0197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.097+0000 7fd6e56c5640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6e0103290 0x7fd6e019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e4ec4640 1 -- 172.21.15.87:0/3054567111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01028d0 msgr2=0x7fd6e0197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e4ec4640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01028d0 0x7fd6e0197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e4ec4640 1 -- 172.21.15.87:0/3054567111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6e0103290 msgr2=0x7fd6e019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e4ec4640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6e0103290 0x7fd6e019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e4ec4640 1 -- 172.21.15.87:0/3054567111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e019f620 con 0x7fd6e01016e0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6dffff640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01028d0 0x7fd6e0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e4ec4640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 0x7fd6e0197a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800e9c0 tx=0x7fd6d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d800cd80 con 0x7fd6e01016e0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd6d80035f0 con 0x7fd6e01016e0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e019f910 con 0x7fd6e01016e0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8010690 con 0x7fd6e01016e0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.098+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e019fde0 con 0x7fd6e01016e0 2024-03-21T08:55:14.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.099+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0108a00 con 0x7fd6e01016e0 2024-03-21T08:55:14.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.100+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd6d80037a0 con 0x7fd6e01016e0 2024-03-21T08:55:14.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.100+0000 7fd6ddffb640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd6cc0a3030 0x7fd6cc0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.101+0000 7fd6dffff640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd6cc0a3030 0x7fd6cc0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.101+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd6d80cf8f0 con 0x7fd6e01016e0 2024-03-21T08:55:14.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.101+0000 7fd6dffff640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd6cc0a3030 0x7fd6cc0a54f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006010 tx=0x7fd6d00325b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:14.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.105+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd6d809c670 con 0x7fd6e01016e0 2024-03-21T08:55:14.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] up:active 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:13 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1086583830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] up:active 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:13 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1086583830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.278+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6e0061ae0 con 0x7fd6cc0a3030 2024-03-21T08:55:14.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.285+0000 7fd6ddffb640 1 -- 172.21.15.87:0/3054567111 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258933 (secure 0 0 0) 0x7fd6e0061ae0 con 0x7fd6cc0a3030 2024-03-21T08:55:14.285 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:14.287 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.288+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd6cc0a3030 msgr2=0x7fd6cc0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.288+0000 7fd6e7128640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd6cc0a3030 0x7fd6cc0a54f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0006010 tx=0x7fd6d00325b0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.288+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 msgr2=0x7fd6e0197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.288+0000 7fd6e7128640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 0x7fd6e0197a00 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd6d800e9c0 tx=0x7fd6d800ee80 comp rx=0 tx=0).stop 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.288+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 shutdown_connections 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd6cc0a3030 0x7fd6cc0a54f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd6e0103290 0x7fd6e019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd6e01028d0 0x7fd6e0197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 --2- 172.21.15.87:0/3054567111 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd6e01016e0 0x7fd6e0197a00 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 >> 172.21.15.87:0/3054567111 conn(0x7fd6e00fcc50 msgr2=0x7fd6e0105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:14.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 shutdown_connections 2024-03-21T08:55:14.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.289+0000 7fd6e7128640 1 -- 172.21.15.87:0/3054567111 wait complete. 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: from='client.14760 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: pgmap v293: 97 pgs: 97 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 7.5 KiB/s wr, 21 op/s 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] up:active 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: fsmap cephfs:5 2 up:standby-replay 5 up:standby 5 up:active 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: from='client.14772 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:14.295 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:13 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1086583830' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:55:14.296 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-03-21T08:55:12.854126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597807,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":135,"num_read_kb":116,"num_write":314,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":244,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356208,"kb_used_data":11176,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585616,"statfs":{"total":1151940427776,"available":1151575670784,"internally_reserved":0,"allocated":11444224,"data_stored":7078105,"data_compressed":106983,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7439,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":133,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.638269"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286204+0000","last_change":"2024-03-21T08:55:07.542089+0000","last_active":"2024-03-21T08:55:09.286204+0000","last_peered":"2024-03-21T08:55:09.286204+0000","last_clean":"2024-03-21T08:55:09.286204+0000","last_became_active":"2024-03-21T08:55:07.541790+0000","last_became_peered":"2024-03-21T08:55:07.541790+0000","last_unstale":"2024-03-21T08:55:09.286204+0000","last_undegraded":"2024-03-21T08:55:09.286204+0000","last_fullsized":"2024-03-21T08:55:09.286204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:57:10.863345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.544909+0000","last_change":"2024-03-21T08:55:07.543892+0000","last_active":"2024-03-21T08:55:07.544909+0000","last_peered":"2024-03-21T08:55:07.544909+0000","last_clean":"2024-03-21T08:55:07.544909+0000","last_became_active":"2024-03-21T08:55:07.541230+0000","last_became_peered":"2024-03-21T08:55:07.541230+0000","last_unstale":"2024-03-21T08:55:07.544909+0000","last_undegraded":"2024-03-21T08:55:07.544909+0000","last_fullsized":"2024-03-21T08:55:07.544909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:17.049460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.540168+0000","last_change":"2024-03-21T08:55:07.537095+0000","last_active":"2024-03-21T08:55:07.540168+0000","last_peered":"2024-03-21T08:55:07.540168+0000","last_clean":"2024-03-21T08:55:07.540168+0000","last_became_active":"2024-03-21T08:55:07.536784+0000","last_became_peered":"2024-03-21T08:55:07.536784+0000","last_unstale":"2024-03-21T08:55:07.540168+0000","last_undegraded":"2024-03-21T08:55:07.540168+0000","last_fullsized":"2024-03-21T08:55:07.540168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:25:28.619856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529236+0000","last_change":"2024-03-21T08:55:07.533330+0000","last_active":"2024-03-21T08:55:08.529236+0000","last_peered":"2024-03-21T08:55:08.529236+0000","last_clean":"2024-03-21T08:55:08.529236+0000","last_became_active":"2024-03-21T08:55:07.532840+0000","last_became_peered":"2024-03-21T08:55:07.532840+0000","last_unstale":"2024-03-21T08:55:08.529236+0000","last_undegraded":"2024-03-21T08:55:08.529236+0000","last_fullsized":"2024-03-21T08:55:08.529236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:52:36.565830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549900+0000","last_change":"2024-03-21T08:55:07.549259+0000","last_active":"2024-03-21T08:55:07.549900+0000","last_peered":"2024-03-21T08:55:07.549900+0000","last_clean":"2024-03-21T08:55:07.549900+0000","last_became_active":"2024-03-21T08:55:07.548161+0000","last_became_peered":"2024-03-21T08:55:07.548161+0000","last_unstale":"2024-03-21T08:55:07.549900+0000","last_undegraded":"2024-03-21T08:55:07.549900+0000","last_fullsized":"2024-03-21T08:55:07.549900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:47:15.235830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126408+0000","last_change":"2024-03-21T08:55:07.530950+0000","last_active":"2024-03-21T08:55:09.126408+0000","last_peered":"2024-03-21T08:55:09.126408+0000","last_clean":"2024-03-21T08:55:09.126408+0000","last_became_active":"2024-03-21T08:55:07.530582+0000","last_became_peered":"2024-03-21T08:55:07.530582+0000","last_unstale":"2024-03-21T08:55:09.126408+0000","last_undegraded":"2024-03-21T08:55:09.126408+0000","last_fullsized":"2024-03-21T08:55:09.126408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:12.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126210+0000","last_change":"2024-03-21T08:55:07.535591+0000","last_active":"2024-03-21T08:55:09.126210+0000","last_peered":"2024-03-21T08:55:09.126210+0000","last_clean":"2024-03-21T08:55:09.126210+0000","last_became_active":"2024-03-21T08:55:07.535257+0000","last_became_peered":"2024-03-21T08:55:07.535257+0000","last_unstale":"2024-03-21T08:55:09.126210+0000","last_undegraded":"2024-03-21T08:55:09.126210+0000","last_fullsized":"2024-03-21T08:55:09.126210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:39.383653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945766+0000","last_change":"2024-03-21T08:55:07.532165+0000","last_active":"2024-03-21T08:55:08.945766+0000","last_peered":"2024-03-21T08:55:08.945766+0000","last_clean":"2024-03-21T08:55:08.945766+0000","last_became_active":"2024-03-21T08:55:07.531822+0000","last_became_peered":"2024-03-21T08:55:07.531822+0000","last_unstale":"2024-03-21T08:55:08.945766+0000","last_undegraded":"2024-03-21T08:55:08.945766+0000","last_fullsized":"2024-03-21T08:55:08.945766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:56.245424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.528055+0000","last_change":"2024-03-21T08:55:07.526371+0000","last_active":"2024-03-21T08:55:07.528055+0000","last_peered":"2024-03-21T08:55:07.528055+0000","last_clean":"2024-03-21T08:55:07.528055+0000","last_became_active":"2024-03-21T08:55:07.526021+0000","last_became_peered":"2024-03-21T08:55:07.526021+0000","last_unstale":"2024-03-21T08:55:07.528055+0000","last_undegraded":"2024-03-21T08:55:07.528055+0000","last_fullsized":"2024-03-21T08:55:07.528055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:55:10.552953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126764+0000","last_change":"2024-03-21T08:55:07.532227+0000","last_active":"2024-03-21T08:55:09.126764+0000","last_peered":"2024-03-21T08:55:09.126764+0000","last_clean":"2024-03-21T08:55:09.126764+0000","last_became_active":"2024-03-21T08:55:07.531349+0000","last_became_peered":"2024-03-21T08:55:07.531349+0000","last_unstale":"2024-03-21T08:55:09.126764+0000","last_undegraded":"2024-03-21T08:55:09.126764+0000","last_fullsized":"2024-03-21T08:55:09.126764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:07:40.318544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126857+0000","last_change":"2024-03-21T08:55:07.533531+0000","last_active":"2024-03-21T08:55:09.126857+0000","last_peered":"2024-03-21T08:55:09.126857+0000","last_clean":"2024-03-21T08:55:09.126857+0000","last_became_active":"2024-03-21T08:55:07.533166+0000","last_became_peered":"2024-03-21T08:55:07.533166+0000","last_unstale":"2024-03-21T08:55:09.126857+0000","last_undegraded":"2024-03-21T08:55:09.126857+0000","last_fullsized":"2024-03-21T08:55:09.126857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:19:24.912748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529153+0000","last_change":"2024-03-21T08:55:07.533643+0000","last_active":"2024-03-21T08:55:08.529153+0000","last_peered":"2024-03-21T08:55:08.529153+0000","last_clean":"2024-03-21T08:55:08.529153+0000","last_became_active":"2024-03-21T08:55:07.533262+0000","last_became_peered":"2024-03-21T08:55:07.533262+0000","last_unstale":"2024-03-21T08:55:08.529153+0000","last_undegraded":"2024-03-21T08:55:08.529153+0000","last_fullsized":"2024-03-21T08:55:08.529153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:07.641225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126326+0000","last_change":"2024-03-21T08:55:07.524670+0000","last_active":"2024-03-21T08:55:09.126326+0000","last_peered":"2024-03-21T08:55:09.126326+0000","last_clean":"2024-03-21T08:55:09.126326+0000","last_became_active":"2024-03-21T08:55:07.524295+0000","last_became_peered":"2024-03-21T08:55:07.524295+0000","last_unstale":"2024-03-21T08:55:09.126326+0000","last_undegraded":"2024-03-21T08:55:09.126326+0000","last_fullsized":"2024-03-21T08:55:09.126326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:07.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.540004+0000","last_change":"2024-03-21T08:55:07.534401+0000","last_active":"2024-03-21T08:55:07.540004+0000","last_peered":"2024-03-21T08:55:07.540004+0000","last_clean":"2024-03-21T08:55:07.540004+0000","last_became_active":"2024-03-21T08:55:07.534056+0000","last_became_peered":"2024-03-21T08:55:07.534056+0000","last_unstale":"2024-03-21T08:55:07.540004+0000","last_undegraded":"2024-03-21T08:55:07.540004+0000","last_fullsized":"2024-03-21T08:55:07.540004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:13.741149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.539847+0000","last_change":"2024-03-21T08:55:07.535915+0000","last_active":"2024-03-21T08:55:07.539847+0000","last_peered":"2024-03-21T08:55:07.539847+0000","last_clean":"2024-03-21T08:55:07.539847+0000","last_became_active":"2024-03-21T08:55:07.535540+0000","last_became_peered":"2024-03-21T08:55:07.535540+0000","last_unstale":"2024-03-21T08:55:07.539847+0000","last_undegraded":"2024-03-21T08:55:07.539847+0000","last_fullsized":"2024-03-21T08:55:07.539847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:45:02.189358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.543193+0000","last_change":"2024-03-21T08:55:07.541894+0000","last_active":"2024-03-21T08:55:07.543193+0000","last_peered":"2024-03-21T08:55:07.543193+0000","last_clean":"2024-03-21T08:55:07.543193+0000","last_became_active":"2024-03-21T08:55:07.541545+0000","last_became_peered":"2024-03-21T08:55:07.541545+0000","last_unstale":"2024-03-21T08:55:07.543193+0000","last_undegraded":"2024-03-21T08:55:07.543193+0000","last_fullsized":"2024-03-21T08:55:07.543193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:41.033026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286370+0000","last_change":"2024-03-21T08:55:07.541228+0000","last_active":"2024-03-21T08:55:09.286370+0000","last_peered":"2024-03-21T08:55:09.286370+0000","last_clean":"2024-03-21T08:55:09.286370+0000","last_became_active":"2024-03-21T08:55:07.540836+0000","last_became_peered":"2024-03-21T08:55:07.540836+0000","last_unstale":"2024-03-21T08:55:09.286370+0000","last_undegraded":"2024-03-21T08:55:09.286370+0000","last_fullsized":"2024-03-21T08:55:09.286370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:34:09.491557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530699+0000","last_change":"2024-03-21T08:55:07.525778+0000","last_active":"2024-03-21T08:55:08.530699+0000","last_peered":"2024-03-21T08:55:08.530699+0000","last_clean":"2024-03-21T08:55:08.530699+0000","last_became_active":"2024-03-21T08:55:07.525469+0000","last_became_peered":"2024-03-21T08:55:07.525469+0000","last_unstale":"2024-03-21T08:55:08.530699+0000","last_undegraded":"2024-03-21T08:55:08.530699+0000","last_fullsized":"2024-03-21T08:55:08.530699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:41:31.715570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529315+0000","last_change":"2024-03-21T08:55:07.529929+0000","last_active":"2024-03-21T08:55:08.529315+0000","last_peered":"2024-03-21T08:55:08.529315+0000","last_clean":"2024-03-21T08:55:08.529315+0000","last_became_active":"2024-03-21T08:55:07.529615+0000","last_became_peered":"2024-03-21T08:55:07.529615+0000","last_unstale":"2024-03-21T08:55:08.529315+0000","last_undegraded":"2024-03-21T08:55:08.529315+0000","last_fullsized":"2024-03-21T08:55:08.529315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:42:19.543201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530482+0000","last_change":"2024-03-21T08:55:06.239149+0000","last_active":"2024-03-21T08:55:08.530482+0000","last_peered":"2024-03-21T08:55:08.530482+0000","last_clean":"2024-03-21T08:55:08.530482+0000","last_became_active":"2024-03-21T08:55:06.238833+0000","last_became_peered":"2024-03-21T08:55:06.238833+0000","last_unstale":"2024-03-21T08:55:08.530482+0000","last_undegraded":"2024-03-21T08:55:08.530482+0000","last_fullsized":"2024-03-21T08:55:08.530482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:28.779442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946214+0000","last_change":"2024-03-21T08:55:07.533596+0000","last_active":"2024-03-21T08:55:08.946214+0000","last_peered":"2024-03-21T08:55:08.946214+0000","last_clean":"2024-03-21T08:55:08.946214+0000","last_became_active":"2024-03-21T08:55:07.533249+0000","last_became_peered":"2024-03-21T08:55:07.533249+0000","last_unstale":"2024-03-21T08:55:08.946214+0000","last_undegraded":"2024-03-21T08:55:08.946214+0000","last_fullsized":"2024-03-21T08:55:08.946214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:21:40.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946109+0000","last_change":"2024-03-21T08:55:06.227045+0000","last_active":"2024-03-21T08:55:08.946109+0000","last_peered":"2024-03-21T08:55:08.946109+0000","last_clean":"2024-03-21T08:55:08.946109+0000","last_became_active":"2024-03-21T08:55:06.226748+0000","last_became_peered":"2024-03-21T08:55:06.226748+0000","last_unstale":"2024-03-21T08:55:08.946109+0000","last_undegraded":"2024-03-21T08:55:08.946109+0000","last_fullsized":"2024-03-21T08:55:08.946109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:37.677094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.541147+0000","last_change":"2024-03-21T08:55:07.540024+0000","last_active":"2024-03-21T08:55:07.541147+0000","last_peered":"2024-03-21T08:55:07.541147+0000","last_clean":"2024-03-21T08:55:07.541147+0000","last_became_active":"2024-03-21T08:55:07.537574+0000","last_became_peered":"2024-03-21T08:55:07.537574+0000","last_unstale":"2024-03-21T08:55:07.541147+0000","last_undegraded":"2024-03-21T08:55:07.541147+0000","last_fullsized":"2024-03-21T08:55:07.541147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:28.732768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.550396+0000","last_change":"2024-03-21T08:55:06.237261+0000","last_active":"2024-03-21T08:55:07.550396+0000","last_peered":"2024-03-21T08:55:07.550396+0000","last_clean":"2024-03-21T08:55:07.550396+0000","last_became_active":"2024-03-21T08:55:06.236838+0000","last_became_peered":"2024-03-21T08:55:06.236838+0000","last_unstale":"2024-03-21T08:55:07.550396+0000","last_undegraded":"2024-03-21T08:55:07.550396+0000","last_fullsized":"2024-03-21T08:55:07.550396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:59.881075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.530299+0000","last_change":"2024-03-21T08:55:07.529159+0000","last_active":"2024-03-21T08:55:07.530299+0000","last_peered":"2024-03-21T08:55:07.530299+0000","last_clean":"2024-03-21T08:55:07.530299+0000","last_became_active":"2024-03-21T08:55:07.527260+0000","last_became_peered":"2024-03-21T08:55:07.527260+0000","last_unstale":"2024-03-21T08:55:07.530299+0000","last_undegraded":"2024-03-21T08:55:07.530299+0000","last_fullsized":"2024-03-21T08:55:07.530299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:00:30.020729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549501+0000","last_change":"2024-03-21T08:55:06.232959+0000","last_active":"2024-03-21T08:55:07.549501+0000","last_peered":"2024-03-21T08:55:07.549501+0000","last_clean":"2024-03-21T08:55:07.549501+0000","last_became_active":"2024-03-21T08:55:06.232606+0000","last_became_peered":"2024-03-21T08:55:06.232606+0000","last_unstale":"2024-03-21T08:55:07.549501+0000","last_undegraded":"2024-03-21T08:55:07.549501+0000","last_fullsized":"2024-03-21T08:55:07.549501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:27:07.480767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126725+0000","last_change":"2024-03-21T08:55:07.530711+0000","last_active":"2024-03-21T08:55:09.126725+0000","last_peered":"2024-03-21T08:55:09.126725+0000","last_clean":"2024-03-21T08:55:09.126725+0000","last_became_active":"2024-03-21T08:55:07.530374+0000","last_became_peered":"2024-03-21T08:55:07.530374+0000","last_unstale":"2024-03-21T08:55:09.126725+0000","last_undegraded":"2024-03-21T08:55:09.126725+0000","last_fullsized":"2024-03-21T08:55:09.126725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:06.524172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.941262+0000","last_change":"2024-03-21T08:55:06.233934+0000","last_active":"2024-03-21T08:55:09.941262+0000","last_peered":"2024-03-21T08:55:09.941262+0000","last_clean":"2024-03-21T08:55:09.941262+0000","last_became_active":"2024-03-21T08:55:06.233618+0000","last_became_peered":"2024-03-21T08:55:06.233618+0000","last_unstale":"2024-03-21T08:55:09.941262+0000","last_undegraded":"2024-03-21T08:55:09.941262+0000","last_fullsized":"2024-03-21T08:55:09.941262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:08.769883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.548183+0000","last_change":"2024-03-21T08:55:07.546411+0000","last_active":"2024-03-21T08:55:07.548183+0000","last_peered":"2024-03-21T08:55:07.548183+0000","last_clean":"2024-03-21T08:55:07.548183+0000","last_became_active":"2024-03-21T08:55:07.546051+0000","last_became_peered":"2024-03-21T08:55:07.546051+0000","last_unstale":"2024-03-21T08:55:07.548183+0000","last_undegraded":"2024-03-21T08:55:07.548183+0000","last_fullsized":"2024-03-21T08:55:07.548183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:54:28.820026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126250+0000","last_change":"2024-03-21T08:55:06.226791+0000","last_active":"2024-03-21T08:55:09.126250+0000","last_peered":"2024-03-21T08:55:09.126250+0000","last_clean":"2024-03-21T08:55:09.126250+0000","last_became_active":"2024-03-21T08:55:06.226438+0000","last_became_peered":"2024-03-21T08:55:06.226438+0000","last_unstale":"2024-03-21T08:55:09.126250+0000","last_undegraded":"2024-03-21T08:55:09.126250+0000","last_fullsized":"2024-03-21T08:55:09.126250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:15:03.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126881+0000","last_change":"2024-03-21T08:55:07.529623+0000","last_active":"2024-03-21T08:55:09.126881+0000","last_peered":"2024-03-21T08:55:09.126881+0000","last_clean":"2024-03-21T08:55:09.126881+0000","last_became_active":"2024-03-21T08:55:07.529293+0000","last_became_peered":"2024-03-21T08:55:07.529293+0000","last_unstale":"2024-03-21T08:55:09.126881+0000","last_undegraded":"2024-03-21T08:55:09.126881+0000","last_fullsized":"2024-03-21T08:55:09.126881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:43.265861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.944837+0000","last_change":"2024-03-21T08:55:06.236381+0000","last_active":"2024-03-21T08:55:09.944837+0000","last_peered":"2024-03-21T08:55:09.944837+0000","last_clean":"2024-03-21T08:55:09.944837+0000","last_became_active":"2024-03-21T08:55:06.236071+0000","last_became_peered":"2024-03-21T08:55:06.236071+0000","last_unstale":"2024-03-21T08:55:09.944837+0000","last_undegraded":"2024-03-21T08:55:09.944837+0000","last_fullsized":"2024-03-21T08:55:09.944837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:13.338191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945840+0000","last_change":"2024-03-21T08:55:07.529637+0000","last_active":"2024-03-21T08:55:08.945840+0000","last_peered":"2024-03-21T08:55:08.945840+0000","last_clean":"2024-03-21T08:55:08.945840+0000","last_became_active":"2024-03-21T08:55:07.527536+0000","last_became_peered":"2024-03-21T08:55:07.527536+0000","last_unstale":"2024-03-21T08:55:08.945840+0000","last_undegraded":"2024-03-21T08:55:08.945840+0000","last_fullsized":"2024-03-21T08:55:08.945840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:15:58.623055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.553938+0000","last_change":"2024-03-21T08:55:06.233121+0000","last_active":"2024-03-21T08:55:09.553938+0000","last_peered":"2024-03-21T08:55:09.553938+0000","last_clean":"2024-03-21T08:55:09.553938+0000","last_became_active":"2024-03-21T08:55:06.232772+0000","last_became_peered":"2024-03-21T08:55:06.232772+0000","last_unstale":"2024-03-21T08:55:09.553938+0000","last_undegraded":"2024-03-21T08:55:09.553938+0000","last_fullsized":"2024-03-21T08:55:09.553938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:05.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530394+0000","last_change":"2024-03-21T08:55:07.527483+0000","last_active":"2024-03-21T08:55:08.530394+0000","last_peered":"2024-03-21T08:55:08.530394+0000","last_clean":"2024-03-21T08:55:08.530394+0000","last_became_active":"2024-03-21T08:55:07.527038+0000","last_became_peered":"2024-03-21T08:55:07.527038+0000","last_unstale":"2024-03-21T08:55:08.530394+0000","last_undegraded":"2024-03-21T08:55:08.530394+0000","last_fullsized":"2024-03-21T08:55:08.530394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:27:13.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.554813+0000","last_change":"2024-03-21T08:55:06.233677+0000","last_active":"2024-03-21T08:55:09.554813+0000","last_peered":"2024-03-21T08:55:09.554813+0000","last_clean":"2024-03-21T08:55:09.554813+0000","last_became_active":"2024-03-21T08:55:06.233312+0000","last_became_peered":"2024-03-21T08:55:06.233312+0000","last_unstale":"2024-03-21T08:55:09.554813+0000","last_undegraded":"2024-03-21T08:55:09.554813+0000","last_fullsized":"2024-03-21T08:55:09.554813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:51:53.894648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.529152+0000","last_change":"2024-03-21T08:55:07.528174+0000","last_active":"2024-03-21T08:55:07.529152+0000","last_peered":"2024-03-21T08:55:07.529152+0000","last_clean":"2024-03-21T08:55:07.529152+0000","last_became_active":"2024-03-21T08:55:07.527313+0000","last_became_peered":"2024-03-21T08:55:07.527313+0000","last_unstale":"2024-03-21T08:55:07.529152+0000","last_undegraded":"2024-03-21T08:55:07.529152+0000","last_fullsized":"2024-03-21T08:55:07.529152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:53:47.876242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.962858+0000","last_change":"2024-03-21T08:55:06.237324+0000","last_active":"2024-03-21T08:55:11.962858+0000","last_peered":"2024-03-21T08:55:11.962858+0000","last_clean":"2024-03-21T08:55:11.962858+0000","last_became_active":"2024-03-21T08:55:06.235319+0000","last_became_peered":"2024-03-21T08:55:06.235319+0000","last_unstale":"2024-03-21T08:55:11.962858+0000","last_undegraded":"2024-03-21T08:55:11.962858+0000","last_fullsized":"2024-03-21T08:55:11.962858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:05:32.872612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.544820+0000","last_change":"2024-03-21T08:55:07.540552+0000","last_active":"2024-03-21T08:55:07.544820+0000","last_peered":"2024-03-21T08:55:07.544820+0000","last_clean":"2024-03-21T08:55:07.544820+0000","last_became_active":"2024-03-21T08:55:07.540211+0000","last_became_peered":"2024-03-21T08:55:07.540211+0000","last_unstale":"2024-03-21T08:55:07.544820+0000","last_undegraded":"2024-03-21T08:55:07.544820+0000","last_fullsized":"2024-03-21T08:55:07.544820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:00:46.298334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.940125+0000","last_change":"2024-03-21T08:55:06.228420+0000","last_active":"2024-03-21T08:55:09.940125+0000","last_peered":"2024-03-21T08:55:09.940125+0000","last_clean":"2024-03-21T08:55:09.940125+0000","last_became_active":"2024-03-21T08:55:06.228037+0000","last_became_peered":"2024-03-21T08:55:06.228037+0000","last_unstale":"2024-03-21T08:55:09.940125+0000","last_undegraded":"2024-03-21T08:55:09.940125+0000","last_fullsized":"2024-03-21T08:55:09.940125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:22.437321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.534498+0000","last_change":"2024-03-21T08:55:07.534053+0000","last_active":"2024-03-21T08:55:07.534498+0000","last_peered":"2024-03-21T08:55:07.534498+0000","last_clean":"2024-03-21T08:55:07.534498+0000","last_became_active":"2024-03-21T08:55:07.533404+0000","last_became_peered":"2024-03-21T08:55:07.533404+0000","last_unstale":"2024-03-21T08:55:07.534498+0000","last_undegraded":"2024-03-21T08:55:07.534498+0000","last_fullsized":"2024-03-21T08:55:07.534498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:09:11.414407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.554423+0000","last_change":"2024-03-21T08:55:06.232681+0000","last_active":"2024-03-21T08:55:09.554423+0000","last_peered":"2024-03-21T08:55:09.554423+0000","last_clean":"2024-03-21T08:55:09.554423+0000","last_became_active":"2024-03-21T08:55:06.232109+0000","last_became_peered":"2024-03-21T08:55:06.232109+0000","last_unstale":"2024-03-21T08:55:09.554423+0000","last_undegraded":"2024-03-21T08:55:09.554423+0000","last_fullsized":"2024-03-21T08:55:09.554423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:11.218106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549378+0000","last_change":"2024-03-21T08:55:07.546464+0000","last_active":"2024-03-21T08:55:07.549378+0000","last_peered":"2024-03-21T08:55:07.549378+0000","last_clean":"2024-03-21T08:55:07.549378+0000","last_became_active":"2024-03-21T08:55:07.546111+0000","last_became_peered":"2024-03-21T08:55:07.546111+0000","last_unstale":"2024-03-21T08:55:07.549378+0000","last_undegraded":"2024-03-21T08:55:07.549378+0000","last_fullsized":"2024-03-21T08:55:07.549378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:12.732453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.958692+0000","last_change":"2024-03-21T08:55:06.233157+0000","last_active":"2024-03-21T08:55:11.958692+0000","last_peered":"2024-03-21T08:55:11.958692+0000","last_clean":"2024-03-21T08:55:11.958692+0000","last_became_active":"2024-03-21T08:55:06.232302+0000","last_became_peered":"2024-03-21T08:55:06.232302+0000","last_unstale":"2024-03-21T08:55:11.958692+0000","last_undegraded":"2024-03-21T08:55:11.958692+0000","last_fullsized":"2024-03-21T08:55:11.958692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:47.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126048+0000","last_change":"2024-03-21T08:55:07.531265+0000","last_active":"2024-03-21T08:55:09.126048+0000","last_peered":"2024-03-21T08:55:09.126048+0000","last_clean":"2024-03-21T08:55:09.126048+0000","last_became_active":"2024-03-21T08:55:07.530913+0000","last_became_peered":"2024-03-21T08:55:07.530913+0000","last_unstale":"2024-03-21T08:55:09.126048+0000","last_undegraded":"2024-03-21T08:55:09.126048+0000","last_fullsized":"2024-03-21T08:55:09.126048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:34.171463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.553830+0000","last_change":"2024-03-21T08:55:06.233411+0000","last_active":"2024-03-21T08:55:09.553830+0000","last_peered":"2024-03-21T08:55:09.553830+0000","last_clean":"2024-03-21T08:55:09.553830+0000","last_became_active":"2024-03-21T08:55:06.233031+0000","last_became_peered":"2024-03-21T08:55:06.233031+0000","last_unstale":"2024-03-21T08:55:09.553830+0000","last_undegraded":"2024-03-21T08:55:09.553830+0000","last_fullsized":"2024-03-21T08:55:09.553830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:52:07.258184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126818+0000","last_change":"2024-03-21T08:55:07.531557+0000","last_active":"2024-03-21T08:55:09.126818+0000","last_peered":"2024-03-21T08:55:09.126818+0000","last_clean":"2024-03-21T08:55:09.126818+0000","last_became_active":"2024-03-21T08:55:07.531183+0000","last_became_peered":"2024-03-21T08:55:07.531183+0000","last_unstale":"2024-03-21T08:55:09.126818+0000","last_undegraded":"2024-03-21T08:55:09.126818+0000","last_fullsized":"2024-03-21T08:55:09.126818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:14:15.401964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.951629+0000","last_change":"2024-03-21T08:55:06.230587+0000","last_active":"2024-03-21T08:55:10.951629+0000","last_peered":"2024-03-21T08:55:10.951629+0000","last_clean":"2024-03-21T08:55:10.951629+0000","last_became_active":"2024-03-21T08:55:06.230201+0000","last_became_peered":"2024-03-21T08:55:06.230201+0000","last_unstale":"2024-03-21T08:55:10.951629+0000","last_undegraded":"2024-03-21T08:55:10.951629+0000","last_fullsized":"2024-03-21T08:55:10.951629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:01.954850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126330+0000","last_change":"2024-03-21T08:55:07.532592+0000","last_active":"2024-03-21T08:55:09.126330+0000","last_peered":"2024-03-21T08:55:09.126330+0000","last_clean":"2024-03-21T08:55:09.126330+0000","last_became_active":"2024-03-21T08:55:07.532210+0000","last_became_peered":"2024-03-21T08:55:07.532210+0000","last_unstale":"2024-03-21T08:55:09.126330+0000","last_undegraded":"2024-03-21T08:55:09.126330+0000","last_fullsized":"2024-03-21T08:55:09.126330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:59.775085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.550788+0000","last_change":"2024-03-21T08:55:06.229096+0000","last_active":"2024-03-21T08:55:07.550788+0000","last_peered":"2024-03-21T08:55:07.550788+0000","last_clean":"2024-03-21T08:55:07.550788+0000","last_became_active":"2024-03-21T08:55:06.228772+0000","last_became_peered":"2024-03-21T08:55:06.228772+0000","last_unstale":"2024-03-21T08:55:07.550788+0000","last_undegraded":"2024-03-21T08:55:07.550788+0000","last_fullsized":"2024-03-21T08:55:07.550788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:05:31.017977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126275+0000","last_change":"2024-03-21T08:55:07.527968+0000","last_active":"2024-03-21T08:55:09.126275+0000","last_peered":"2024-03-21T08:55:09.126275+0000","last_clean":"2024-03-21T08:55:09.126275+0000","last_became_active":"2024-03-21T08:55:07.523326+0000","last_became_peered":"2024-03-21T08:55:07.523326+0000","last_unstale":"2024-03-21T08:55:09.126275+0000","last_undegraded":"2024-03-21T08:55:09.126275+0000","last_fullsized":"2024-03-21T08:55:09.126275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:01:22.903576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549492+0000","last_change":"2024-03-21T08:55:06.225599+0000","last_active":"2024-03-21T08:55:07.549492+0000","last_peered":"2024-03-21T08:55:07.549492+0000","last_clean":"2024-03-21T08:55:07.549492+0000","last_became_active":"2024-03-21T08:55:06.225205+0000","last_became_peered":"2024-03-21T08:55:06.225205+0000","last_unstale":"2024-03-21T08:55:07.549492+0000","last_undegraded":"2024-03-21T08:55:07.549492+0000","last_fullsized":"2024-03-21T08:55:07.549492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:07.286408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286116+0000","last_change":"2024-03-21T08:55:07.532577+0000","last_active":"2024-03-21T08:55:09.286116+0000","last_peered":"2024-03-21T08:55:09.286116+0000","last_clean":"2024-03-21T08:55:09.286116+0000","last_became_active":"2024-03-21T08:55:07.529015+0000","last_became_peered":"2024-03-21T08:55:07.529015+0000","last_unstale":"2024-03-21T08:55:09.286116+0000","last_undegraded":"2024-03-21T08:55:09.286116+0000","last_fullsized":"2024-03-21T08:55:09.286116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:24.045732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530544+0000","last_change":"2024-03-21T08:55:06.238430+0000","last_active":"2024-03-21T08:55:08.530544+0000","last_peered":"2024-03-21T08:55:08.530544+0000","last_clean":"2024-03-21T08:55:08.530544+0000","last_became_active":"2024-03-21T08:55:06.238128+0000","last_became_peered":"2024-03-21T08:55:06.238128+0000","last_unstale":"2024-03-21T08:55:08.530544+0000","last_undegraded":"2024-03-21T08:55:08.530544+0000","last_fullsized":"2024-03-21T08:55:08.530544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:49.839546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126810+0000","last_change":"2024-03-21T08:55:07.540750+0000","last_active":"2024-03-21T08:55:09.126810+0000","last_peered":"2024-03-21T08:55:09.126810+0000","last_clean":"2024-03-21T08:55:09.126810+0000","last_became_active":"2024-03-21T08:55:07.540358+0000","last_became_peered":"2024-03-21T08:55:07.540358+0000","last_unstale":"2024-03-21T08:55:09.126810+0000","last_undegraded":"2024-03-21T08:55:09.126810+0000","last_fullsized":"2024-03-21T08:55:09.126810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:37.590268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.963020+0000","last_change":"2024-03-21T08:55:06.236387+0000","last_active":"2024-03-21T08:55:11.963020+0000","last_peered":"2024-03-21T08:55:11.963020+0000","last_clean":"2024-03-21T08:55:11.963020+0000","last_became_active":"2024-03-21T08:55:06.234603+0000","last_became_peered":"2024-03-21T08:55:06.234603+0000","last_unstale":"2024-03-21T08:55:11.963020+0000","last_undegraded":"2024-03-21T08:55:11.963020+0000","last_fullsized":"2024-03-21T08:55:11.963020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:10:28.921679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126959+0000","last_change":"2024-03-21T08:55:07.539011+0000","last_active":"2024-03-21T08:55:09.126959+0000","last_peered":"2024-03-21T08:55:09.126959+0000","last_clean":"2024-03-21T08:55:09.126959+0000","last_became_active":"2024-03-21T08:55:07.538694+0000","last_became_peered":"2024-03-21T08:55:07.538694+0000","last_unstale":"2024-03-21T08:55:09.126959+0000","last_undegraded":"2024-03-21T08:55:09.126959+0000","last_fullsized":"2024-03-21T08:55:09.126959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:52.746409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126303+0000","last_change":"2024-03-21T08:55:07.531605+0000","last_active":"2024-03-21T08:55:09.126303+0000","last_peered":"2024-03-21T08:55:09.126303+0000","last_clean":"2024-03-21T08:55:09.126303+0000","last_became_active":"2024-03-21T08:55:07.530207+0000","last_became_peered":"2024-03-21T08:55:07.530207+0000","last_unstale":"2024-03-21T08:55:09.126303+0000","last_undegraded":"2024-03-21T08:55:09.126303+0000","last_fullsized":"2024-03-21T08:55:09.126303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:58.805184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.945664+0000","last_change":"2024-03-21T08:55:06.228108+0000","last_active":"2024-03-21T08:55:09.945664+0000","last_peered":"2024-03-21T08:55:09.945664+0000","last_clean":"2024-03-21T08:55:09.945664+0000","last_became_active":"2024-03-21T08:55:06.227281+0000","last_became_peered":"2024-03-21T08:55:06.227281+0000","last_unstale":"2024-03-21T08:55:09.945664+0000","last_undegraded":"2024-03-21T08:55:09.945664+0000","last_fullsized":"2024-03-21T08:55:09.945664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:02.026031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946052+0000","last_change":"2024-03-21T08:55:07.531090+0000","last_active":"2024-03-21T08:55:08.946052+0000","last_peered":"2024-03-21T08:55:08.946052+0000","last_clean":"2024-03-21T08:55:08.946052+0000","last_became_active":"2024-03-21T08:55:07.530769+0000","last_became_peered":"2024-03-21T08:55:07.530769+0000","last_unstale":"2024-03-21T08:55:08.946052+0000","last_undegraded":"2024-03-21T08:55:08.946052+0000","last_fullsized":"2024-03-21T08:55:08.946052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:49:51.950743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.953349+0000","last_change":"2024-03-21T08:55:06.228583+0000","last_active":"2024-03-21T08:55:10.953349+0000","last_peered":"2024-03-21T08:55:10.953349+0000","last_clean":"2024-03-21T08:55:10.953349+0000","last_became_active":"2024-03-21T08:55:06.228222+0000","last_became_peered":"2024-03-21T08:55:06.228222+0000","last_unstale":"2024-03-21T08:55:10.953349+0000","last_undegraded":"2024-03-21T08:55:10.953349+0000","last_fullsized":"2024-03-21T08:55:10.953349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:13:28.254775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126326+0000","last_change":"2024-03-21T08:53:52.956820+0000","last_active":"2024-03-21T08:55:09.126326+0000","last_peered":"2024-03-21T08:55:09.126326+0000","last_clean":"2024-03-21T08:55:09.126326+0000","last_became_active":"2024-03-21T08:53:52.956322+0000","last_became_peered":"2024-03-21T08:53:52.956322+0000","last_unstale":"2024-03-21T08:55:09.126326+0000","last_undegraded":"2024-03-21T08:55:09.126326+0000","last_fullsized":"2024-03-21T08:55:09.126326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_clean_scrub_stamp":"2024-03-21T08:50:27.622697+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:55.824313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.125902+0000","last_change":"2024-03-21T08:55:07.529927+0000","last_active":"2024-03-21T08:55:09.125902+0000","last_peered":"2024-03-21T08:55:09.125902+0000","last_clean":"2024-03-21T08:55:09.125902+0000","last_became_active":"2024-03-21T08:55:07.529588+0000","last_became_peered":"2024-03-21T08:55:07.529588+0000","last_unstale":"2024-03-21T08:55:09.125902+0000","last_undegraded":"2024-03-21T08:55:09.125902+0000","last_fullsized":"2024-03-21T08:55:09.125902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:11:04.122448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.945339+0000","last_change":"2024-03-21T08:55:06.236496+0000","last_active":"2024-03-21T08:55:09.945339+0000","last_peered":"2024-03-21T08:55:09.945339+0000","last_clean":"2024-03-21T08:55:09.945339+0000","last_became_active":"2024-03-21T08:55:06.234447+0000","last_became_peered":"2024-03-21T08:55:06.234447+0000","last_unstale":"2024-03-21T08:55:09.945339+0000","last_undegraded":"2024-03-21T08:55:09.945339+0000","last_fullsized":"2024-03-21T08:55:09.945339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:46:55.322330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.523352+0000","last_change":"2024-03-21T08:55:06.232367+0000","last_active":"2024-03-21T08:55:07.523352+0000","last_peered":"2024-03-21T08:55:07.523352+0000","last_clean":"2024-03-21T08:55:07.523352+0000","last_became_active":"2024-03-21T08:55:06.229741+0000","last_became_peered":"2024-03-21T08:55:06.229741+0000","last_unstale":"2024-03-21T08:55:07.523352+0000","last_undegraded":"2024-03-21T08:55:07.523352+0000","last_fullsized":"2024-03-21T08:55:07.523352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:51.644926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126261+0000","last_change":"2024-03-21T08:55:07.527366+0000","last_active":"2024-03-21T08:55:09.126261+0000","last_peered":"2024-03-21T08:55:09.126261+0000","last_clean":"2024-03-21T08:55:09.126261+0000","last_became_active":"2024-03-21T08:55:07.527033+0000","last_became_peered":"2024-03-21T08:55:07.527033+0000","last_unstale":"2024-03-21T08:55:09.126261+0000","last_undegraded":"2024-03-21T08:55:09.126261+0000","last_fullsized":"2024-03-21T08:55:09.126261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:07:38.400820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530615+0000","last_change":"2024-03-21T08:55:06.234078+0000","last_active":"2024-03-21T08:55:08.530615+0000","last_peered":"2024-03-21T08:55:08.530615+0000","last_clean":"2024-03-21T08:55:08.530615+0000","last_became_active":"2024-03-21T08:55:06.233704+0000","last_became_peered":"2024-03-21T08:55:06.233704+0000","last_unstale":"2024-03-21T08:55:08.530615+0000","last_undegraded":"2024-03-21T08:55:08.530615+0000","last_fullsized":"2024-03-21T08:55:08.530615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:08:00.957142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946235+0000","last_change":"2024-03-21T08:55:07.529668+0000","last_active":"2024-03-21T08:55:08.946235+0000","last_peered":"2024-03-21T08:55:08.946235+0000","last_clean":"2024-03-21T08:55:08.946235+0000","last_became_active":"2024-03-21T08:55:07.528560+0000","last_became_peered":"2024-03-21T08:55:07.528560+0000","last_unstale":"2024-03-21T08:55:08.946235+0000","last_undegraded":"2024-03-21T08:55:08.946235+0000","last_fullsized":"2024-03-21T08:55:08.946235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:49.671107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.961748+0000","last_change":"2024-03-21T08:55:06.241169+0000","last_active":"2024-03-21T08:55:11.961748+0000","last_peered":"2024-03-21T08:55:11.961748+0000","last_clean":"2024-03-21T08:55:11.961748+0000","last_became_active":"2024-03-21T08:55:06.240740+0000","last_became_peered":"2024-03-21T08:55:06.240740+0000","last_unstale":"2024-03-21T08:55:11.961748+0000","last_undegraded":"2024-03-21T08:55:11.961748+0000","last_fullsized":"2024-03-21T08:55:11.961748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:31.327983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.546595+0000","last_change":"2024-03-21T08:55:07.544243+0000","last_active":"2024-03-21T08:55:07.546595+0000","last_peered":"2024-03-21T08:55:07.546595+0000","last_clean":"2024-03-21T08:55:07.546595+0000","last_became_active":"2024-03-21T08:55:07.543827+0000","last_became_peered":"2024-03-21T08:55:07.543827+0000","last_unstale":"2024-03-21T08:55:07.546595+0000","last_undegraded":"2024-03-21T08:55:07.546595+0000","last_fullsized":"2024-03-21T08:55:07.546595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:30:07.859809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.512267+0000","last_change":"2024-03-21T08:55:06.236769+0000","last_active":"2024-03-21T08:55:07.512267+0000","last_peered":"2024-03-21T08:55:07.512267+0000","last_clean":"2024-03-21T08:55:07.512267+0000","last_became_active":"2024-03-21T08:55:06.236426+0000","last_became_peered":"2024-03-21T08:55:06.236426+0000","last_unstale":"2024-03-21T08:55:07.512267+0000","last_undegraded":"2024-03-21T08:55:07.512267+0000","last_fullsized":"2024-03-21T08:55:07.512267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:51.833976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.545073+0000","last_change":"2024-03-21T08:55:07.540974+0000","last_active":"2024-03-21T08:55:07.545073+0000","last_peered":"2024-03-21T08:55:07.545073+0000","last_clean":"2024-03-21T08:55:07.545073+0000","last_became_active":"2024-03-21T08:55:07.540246+0000","last_became_peered":"2024-03-21T08:55:07.540246+0000","last_unstale":"2024-03-21T08:55:07.545073+0000","last_undegraded":"2024-03-21T08:55:07.545073+0000","last_fullsized":"2024-03-21T08:55:07.545073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:27.624121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.953272+0000","last_change":"2024-03-21T08:55:06.231544+0000","last_active":"2024-03-21T08:55:10.953272+0000","last_peered":"2024-03-21T08:55:10.953272+0000","last_clean":"2024-03-21T08:55:10.953272+0000","last_became_active":"2024-03-21T08:55:06.231238+0000","last_became_peered":"2024-03-21T08:55:06.231238+0000","last_unstale":"2024-03-21T08:55:10.953272+0000","last_undegraded":"2024-03-21T08:55:10.953272+0000","last_fullsized":"2024-03-21T08:55:10.953272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:34.831596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.285957+0000","last_change":"2024-03-21T08:55:07.541189+0000","last_active":"2024-03-21T08:55:09.285957+0000","last_peered":"2024-03-21T08:55:09.285957+0000","last_clean":"2024-03-21T08:55:09.285957+0000","last_became_active":"2024-03-21T08:55:07.540819+0000","last_became_peered":"2024-03-21T08:55:07.540819+0000","last_unstale":"2024-03-21T08:55:09.285957+0000","last_undegraded":"2024-03-21T08:55:09.285957+0000","last_fullsized":"2024-03-21T08:55:09.285957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:21:43.879895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.953117+0000","last_change":"2024-03-21T08:55:06.231164+0000","last_active":"2024-03-21T08:55:10.953117+0000","last_peered":"2024-03-21T08:55:10.953117+0000","last_clean":"2024-03-21T08:55:10.953117+0000","last_became_active":"2024-03-21T08:55:06.230782+0000","last_became_peered":"2024-03-21T08:55:06.230782+0000","last_unstale":"2024-03-21T08:55:10.953117+0000","last_undegraded":"2024-03-21T08:55:10.953117+0000","last_fullsized":"2024-03-21T08:55:10.953117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:26:56.744957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945989+0000","last_change":"2024-03-21T08:55:07.544379+0000","last_active":"2024-03-21T08:55:08.945989+0000","last_peered":"2024-03-21T08:55:08.945989+0000","last_clean":"2024-03-21T08:55:08.945989+0000","last_became_active":"2024-03-21T08:55:07.541643+0000","last_became_peered":"2024-03-21T08:55:07.541643+0000","last_unstale":"2024-03-21T08:55:08.945989+0000","last_undegraded":"2024-03-21T08:55:08.945989+0000","last_fullsized":"2024-03-21T08:55:08.945989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:54:19.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.550522+0000","last_change":"2024-03-21T08:55:06.229859+0000","last_active":"2024-03-21T08:55:07.550522+0000","last_peered":"2024-03-21T08:55:07.550522+0000","last_clean":"2024-03-21T08:55:07.550522+0000","last_became_active":"2024-03-21T08:55:06.229532+0000","last_became_peered":"2024-03-21T08:55:06.229532+0000","last_unstale":"2024-03-21T08:55:07.550522+0000","last_undegraded":"2024-03-21T08:55:07.550522+0000","last_fullsized":"2024-03-21T08:55:07.550522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:39.615417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126847+0000","last_change":"2024-03-21T08:55:07.530619+0000","last_active":"2024-03-21T08:55:09.126847+0000","last_peered":"2024-03-21T08:55:09.126847+0000","last_clean":"2024-03-21T08:55:09.126847+0000","last_became_active":"2024-03-21T08:55:07.530293+0000","last_became_peered":"2024-03-21T08:55:07.530293+0000","last_unstale":"2024-03-21T08:55:09.126847+0000","last_undegraded":"2024-03-21T08:55:09.126847+0000","last_fullsized":"2024-03-21T08:55:09.126847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:44:56.242621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.951888+0000","last_change":"2024-03-21T08:55:06.241581+0000","last_active":"2024-03-21T08:55:10.951888+0000","last_peered":"2024-03-21T08:55:10.951888+0000","last_clean":"2024-03-21T08:55:10.951888+0000","last_became_active":"2024-03-21T08:55:06.241162+0000","last_became_peered":"2024-03-21T08:55:06.241162+0000","last_unstale":"2024-03-21T08:55:10.951888+0000","last_undegraded":"2024-03-21T08:55:10.951888+0000","last_fullsized":"2024-03-21T08:55:10.951888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:15.231840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.127064+0000","last_change":"2024-03-21T08:55:07.532424+0000","last_active":"2024-03-21T08:55:09.127064+0000","last_peered":"2024-03-21T08:55:09.127064+0000","last_clean":"2024-03-21T08:55:09.127064+0000","last_became_active":"2024-03-21T08:55:07.532062+0000","last_became_peered":"2024-03-21T08:55:07.532062+0000","last_unstale":"2024-03-21T08:55:09.127064+0000","last_undegraded":"2024-03-21T08:55:09.127064+0000","last_fullsized":"2024-03-21T08:55:09.127064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:44:07.970387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.936262+0000","last_change":"2024-03-21T08:55:06.232191+0000","last_active":"2024-03-21T08:55:09.936262+0000","last_peered":"2024-03-21T08:55:09.936262+0000","last_clean":"2024-03-21T08:55:09.936262+0000","last_became_active":"2024-03-21T08:55:06.231880+0000","last_became_peered":"2024-03-21T08:55:06.231880+0000","last_unstale":"2024-03-21T08:55:09.936262+0000","last_undegraded":"2024-03-21T08:55:09.936262+0000","last_fullsized":"2024-03-21T08:55:09.936262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:35.434959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126310+0000","last_change":"2024-03-21T08:55:07.538588+0000","last_active":"2024-03-21T08:55:09.126310+0000","last_peered":"2024-03-21T08:55:09.126310+0000","last_clean":"2024-03-21T08:55:09.126310+0000","last_became_active":"2024-03-21T08:55:07.538268+0000","last_became_peered":"2024-03-21T08:55:07.538268+0000","last_unstale":"2024-03-21T08:55:09.126310+0000","last_undegraded":"2024-03-21T08:55:09.126310+0000","last_fullsized":"2024-03-21T08:55:09.126310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:21.591094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.960871+0000","last_change":"2024-03-21T08:55:06.228166+0000","last_active":"2024-03-21T08:55:11.960871+0000","last_peered":"2024-03-21T08:55:11.960871+0000","last_clean":"2024-03-21T08:55:11.960871+0000","last_became_active":"2024-03-21T08:55:06.227852+0000","last_became_peered":"2024-03-21T08:55:06.227852+0000","last_unstale":"2024-03-21T08:55:11.960871+0000","last_undegraded":"2024-03-21T08:55:11.960871+0000","last_fullsized":"2024-03-21T08:55:11.960871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:12.471809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126978+0000","last_change":"2024-03-21T08:55:07.536854+0000","last_active":"2024-03-21T08:55:09.126978+0000","last_peered":"2024-03-21T08:55:09.126978+0000","last_clean":"2024-03-21T08:55:09.126978+0000","last_became_active":"2024-03-21T08:55:07.536241+0000","last_became_peered":"2024-03-21T08:55:07.536241+0000","last_unstale":"2024-03-21T08:55:09.126978+0000","last_undegraded":"2024-03-21T08:55:09.126978+0000","last_fullsized":"2024-03-21T08:55:09.126978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:21.060162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286151+0000","last_change":"2024-03-21T08:55:07.528084+0000","last_active":"2024-03-21T08:55:09.286151+0000","last_peered":"2024-03-21T08:55:09.286151+0000","last_clean":"2024-03-21T08:55:09.286151+0000","last_became_active":"2024-03-21T08:55:07.527675+0000","last_became_peered":"2024-03-21T08:55:07.527675+0000","last_unstale":"2024-03-21T08:55:09.286151+0000","last_undegraded":"2024-03-21T08:55:09.286151+0000","last_fullsized":"2024-03-21T08:55:09.286151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:47:21.646078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529049+0000","last_change":"2024-03-21T08:55:07.542384+0000","last_active":"2024-03-21T08:55:08.529049+0000","last_peered":"2024-03-21T08:55:08.529049+0000","last_clean":"2024-03-21T08:55:08.529049+0000","last_became_active":"2024-03-21T08:55:07.541956+0000","last_became_peered":"2024-03-21T08:55:07.541956+0000","last_unstale":"2024-03-21T08:55:08.529049+0000","last_undegraded":"2024-03-21T08:55:08.529049+0000","last_fullsized":"2024-03-21T08:55:08.529049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:54:47.093433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.528998+0000","last_change":"2024-03-21T08:55:07.536226+0000","last_active":"2024-03-21T08:55:08.528998+0000","last_peered":"2024-03-21T08:55:08.528998+0000","last_clean":"2024-03-21T08:55:08.528998+0000","last_became_active":"2024-03-21T08:55:07.535711+0000","last_became_peered":"2024-03-21T08:55:07.535711+0000","last_unstale":"2024-03-21T08:55:08.528998+0000","last_undegraded":"2024-03-21T08:55:08.528998+0000","last_fullsized":"2024-03-21T08:55:08.528998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:37:29.252051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.540749+0000","last_change":"2024-03-21T08:55:07.536440+0000","last_active":"2024-03-21T08:55:07.540749+0000","last_peered":"2024-03-21T08:55:07.540749+0000","last_clean":"2024-03-21T08:55:07.540749+0000","last_became_active":"2024-03-21T08:55:07.536076+0000","last_became_peered":"2024-03-21T08:55:07.536076+0000","last_unstale":"2024-03-21T08:55:07.540749+0000","last_undegraded":"2024-03-21T08:55:07.540749+0000","last_fullsized":"2024-03-21T08:55:07.540749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:01:29.420502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126426+0000","last_change":"2024-03-21T08:55:07.542634+0000","last_active":"2024-03-21T08:55:09.126426+0000","last_peered":"2024-03-21T08:55:09.126426+0000","last_clean":"2024-03-21T08:55:09.126426+0000","last_became_active":"2024-03-21T08:55:07.542317+0000","last_became_peered":"2024-03-21T08:55:07.542317+0000","last_unstale":"2024-03-21T08:55:09.126426+0000","last_undegraded":"2024-03-21T08:55:09.126426+0000","last_fullsized":"2024-03-21T08:55:09.126426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:31.392606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286203+0000","last_change":"2024-03-21T08:55:07.537660+0000","last_active":"2024-03-21T08:55:09.286203+0000","last_peered":"2024-03-21T08:55:09.286203+0000","last_clean":"2024-03-21T08:55:09.286203+0000","last_became_active":"2024-03-21T08:55:07.537205+0000","last_became_peered":"2024-03-21T08:55:07.537205+0000","last_unstale":"2024-03-21T08:55:09.286203+0000","last_undegraded":"2024-03-21T08:55:09.286203+0000","last_fullsized":"2024-03-21T08:55:09.286203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:39:59.336323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529504+0000","last_change":"2024-03-21T08:55:07.530140+0000","last_active":"2024-03-21T08:55:08.529504+0000","last_peered":"2024-03-21T08:55:08.529504+0000","last_clean":"2024-03-21T08:55:08.529504+0000","last_became_active":"2024-03-21T08:55:07.529373+0000","last_became_peered":"2024-03-21T08:55:07.529373+0000","last_unstale":"2024-03-21T08:55:08.529504+0000","last_undegraded":"2024-03-21T08:55:08.529504+0000","last_fullsized":"2024-03-21T08:55:08.529504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:37.939733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946194+0000","last_change":"2024-03-21T08:55:07.529218+0000","last_active":"2024-03-21T08:55:08.946194+0000","last_peered":"2024-03-21T08:55:08.946194+0000","last_clean":"2024-03-21T08:55:08.946194+0000","last_became_active":"2024-03-21T08:55:07.528898+0000","last_became_peered":"2024-03-21T08:55:07.528898+0000","last_unstale":"2024-03-21T08:55:08.946194+0000","last_undegraded":"2024-03-21T08:55:08.946194+0000","last_fullsized":"2024-03-21T08:55:08.946194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:49.392688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126165+0000","last_change":"2024-03-21T08:55:07.531920+0000","last_active":"2024-03-21T08:55:09.126165+0000","last_peered":"2024-03-21T08:55:09.126165+0000","last_clean":"2024-03-21T08:55:09.126165+0000","last_became_active":"2024-03-21T08:55:07.531266+0000","last_became_peered":"2024-03-21T08:55:07.531266+0000","last_unstale":"2024-03-21T08:55:09.126165+0000","last_undegraded":"2024-03-21T08:55:09.126165+0000","last_fullsized":"2024-03-21T08:55:09.126165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:41:19.838525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945794+0000","last_change":"2024-03-21T08:55:07.530658+0000","last_active":"2024-03-21T08:55:08.945794+0000","last_peered":"2024-03-21T08:55:08.945794+0000","last_clean":"2024-03-21T08:55:08.945794+0000","last_became_active":"2024-03-21T08:55:07.529933+0000","last_became_peered":"2024-03-21T08:55:07.529933+0000","last_unstale":"2024-03-21T08:55:08.945794+0000","last_undegraded":"2024-03-21T08:55:08.945794+0000","last_fullsized":"2024-03-21T08:55:08.945794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:56.698603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.530269+0000","last_change":"2024-03-21T08:55:07.529151+0000","last_active":"2024-03-21T08:55:07.530269+0000","last_peered":"2024-03-21T08:55:07.530269+0000","last_clean":"2024-03-21T08:55:07.530269+0000","last_became_active":"2024-03-21T08:55:07.528511+0000","last_became_peered":"2024-03-21T08:55:07.528511+0000","last_unstale":"2024-03-21T08:55:07.530269+0000","last_undegraded":"2024-03-21T08:55:07.530269+0000","last_fullsized":"2024-03-21T08:55:07.530269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:17.745591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126468+0000","last_change":"2024-03-21T08:55:07.541519+0000","last_active":"2024-03-21T08:55:09.126468+0000","last_peered":"2024-03-21T08:55:09.126468+0000","last_clean":"2024-03-21T08:55:09.126468+0000","last_became_active":"2024-03-21T08:55:07.541171+0000","last_became_peered":"2024-03-21T08:55:07.541171+0000","last_unstale":"2024-03-21T08:55:09.126468+0000","last_undegraded":"2024-03-21T08:55:09.126468+0000","last_fullsized":"2024-03-21T08:55:09.126468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:10:36.249778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286028+0000","last_change":"2024-03-21T08:55:07.538455+0000","last_active":"2024-03-21T08:55:09.286028+0000","last_peered":"2024-03-21T08:55:09.286028+0000","last_clean":"2024-03-21T08:55:09.286028+0000","last_became_active":"2024-03-21T08:55:07.538082+0000","last_became_peered":"2024-03-21T08:55:07.538082+0000","last_unstale":"2024-03-21T08:55:09.286028+0000","last_undegraded":"2024-03-21T08:55:09.286028+0000","last_fullsized":"2024-03-21T08:55:09.286028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:18:42.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280365,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7439,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":133,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":24824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":444207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1035189,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":443597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":435049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1036238,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":443127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":436057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":446990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":444072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1035621,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:55:14.301 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 -- 172.21.15.87:0/965421620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 msgr2=0x7f6d50106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 --2- 172.21.15.87:0/965421620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 0x7f6d50106ec0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6d44009900 tx=0x7f6d4402bc30 comp rx=0 tx=0).stop 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 -- 172.21.15.87:0/965421620 shutdown_connections 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 --2- 172.21.15.87:0/965421620 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d50107400 0x7f6d501097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 --2- 172.21.15.87:0/965421620 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 0x7f6d50106ec0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 --2- 172.21.15.87:0/965421620 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d501021a0 0x7f6d50104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 -- 172.21.15.87:0/965421620 >> 172.21.15.87:0/965421620 conn(0x7f6d500fbb30 msgr2=0x7f6d500fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 -- 172.21.15.87:0/965421620 shutdown_connections 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.440+0000 7f6d55987640 1 -- 172.21.15.87:0/965421620 wait complete. 2024-03-21T08:55:14.439 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 Processor -- start 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 -- start start 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 0x7f6d50101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 0x7f6d50100150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d50107400 0x7f6d50100690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5010d330 con 0x7f6d501021a0 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5010d1b0 con 0x7f6d50107400 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d55987640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5010d4b0 con 0x7f6d50104ad0 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 0x7f6d50101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 0x7f6d50101b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:44946/0 (socket says 172.21.15.87:44946) 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 -- 172.21.15.87:0/1408054402 learned_addr learned my addr 172.21.15.87:0/1408054402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4f7fe640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d50107400 0x7f6d50100690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4e7fc640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 0x7f6d50100150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 -- 172.21.15.87:0/1408054402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 msgr2=0x7f6d50100150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 0x7f6d50100150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 -- 172.21.15.87:0/1408054402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d50107400 msgr2=0x7f6d50100690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d50107400 0x7f6d50100690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.441+0000 7f6d4effd640 1 -- 172.21.15.87:0/1408054402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d50100f20 con 0x7f6d501021a0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.442+0000 7f6d4effd640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 0x7f6d50101b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000ea80 tx=0x7f6d4000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.442+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d4000ce40 con 0x7f6d501021a0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.442+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d501a1b20 con 0x7f6d501021a0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.442+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6d40005020 con 0x7f6d501021a0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.442+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d40011760 con 0x7f6d501021a0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.442+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d501a2040 con 0x7f6d501021a0 2024-03-21T08:55:14.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.443+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d50068490 con 0x7f6d501021a0 2024-03-21T08:55:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.444+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f6d400051d0 con 0x7f6d501021a0 2024-03-21T08:55:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.444+0000 7f6d54985640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d380a2fb0 0x7f6d380a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:14.442 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.444+0000 7f6d4e7fc640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d380a2fb0 0x7f6d380a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.444+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f6d400cf9a0 con 0x7f6d501021a0 2024-03-21T08:55:14.443 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.445+0000 7f6d4e7fc640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d380a2fb0 0x7f6d380a5470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d44000c00 tx=0x7f6d44005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:14.445 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.449+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6d40016030 con 0x7f6d501021a0 2024-03-21T08:55:14.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.621+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d50061ba0 con 0x7f6d380a2fb0 2024-03-21T08:55:14.624 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.628+0000 7f6d54985640 1 -- 172.21.15.87:0/1408054402 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258933 (secure 0 0 0) 0x7f6d50061ba0 con 0x7f6d380a2fb0 2024-03-21T08:55:14.628 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:14.637 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d380a2fb0 msgr2=0x7f6d380a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d380a2fb0 0x7f6d380a5470 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6d44000c00 tx=0x7f6d44005fd0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 msgr2=0x7f6d50101b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 0x7f6d50101b10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f6d4000ea80 tx=0x7f6d4000ef40 comp rx=0 tx=0).stop 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 shutdown_connections 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6d380a2fb0 0x7f6d380a5470 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6d50107400 0x7f6d50100690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6d50104ad0 0x7f6d50100150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 --2- 172.21.15.87:0/1408054402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6d501021a0 0x7f6d50101b10 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 >> 172.21.15.87:0/1408054402 conn(0x7f6d500fbb30 msgr2=0x7f6d50102c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 shutdown_connections 2024-03-21T08:55:14.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:14.631+0000 7f6d55987640 1 -- 172.21.15.87:0/1408054402 wait complete. 2024-03-21T08:55:14.645 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-03-21T08:55:12.854126+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597807,"num_objects":68,"num_object_clones":0,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":0,"num_read":135,"num_read_kb":116,"num_write":314,"num_write_kb":2988,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":256,"ondisk_log_size":256,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":244,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356208,"kb_used_data":11176,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124585616,"statfs":{"total":1151940427776,"available":1151575670784,"internally_reserved":0,"allocated":11444224,"data_stored":7078105,"data_compressed":106983,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":7439,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":133,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.638269"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286204+0000","last_change":"2024-03-21T08:55:07.542089+0000","last_active":"2024-03-21T08:55:09.286204+0000","last_peered":"2024-03-21T08:55:09.286204+0000","last_clean":"2024-03-21T08:55:09.286204+0000","last_became_active":"2024-03-21T08:55:07.541790+0000","last_became_peered":"2024-03-21T08:55:07.541790+0000","last_unstale":"2024-03-21T08:55:09.286204+0000","last_undegraded":"2024-03-21T08:55:09.286204+0000","last_fullsized":"2024-03-21T08:55:09.286204+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:57:10.863345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.544909+0000","last_change":"2024-03-21T08:55:07.543892+0000","last_active":"2024-03-21T08:55:07.544909+0000","last_peered":"2024-03-21T08:55:07.544909+0000","last_clean":"2024-03-21T08:55:07.544909+0000","last_became_active":"2024-03-21T08:55:07.541230+0000","last_became_peered":"2024-03-21T08:55:07.541230+0000","last_unstale":"2024-03-21T08:55:07.544909+0000","last_undegraded":"2024-03-21T08:55:07.544909+0000","last_fullsized":"2024-03-21T08:55:07.544909+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:42:17.049460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.540168+0000","last_change":"2024-03-21T08:55:07.537095+0000","last_active":"2024-03-21T08:55:07.540168+0000","last_peered":"2024-03-21T08:55:07.540168+0000","last_clean":"2024-03-21T08:55:07.540168+0000","last_became_active":"2024-03-21T08:55:07.536784+0000","last_became_peered":"2024-03-21T08:55:07.536784+0000","last_unstale":"2024-03-21T08:55:07.540168+0000","last_undegraded":"2024-03-21T08:55:07.540168+0000","last_fullsized":"2024-03-21T08:55:07.540168+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:25:28.619856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529236+0000","last_change":"2024-03-21T08:55:07.533330+0000","last_active":"2024-03-21T08:55:08.529236+0000","last_peered":"2024-03-21T08:55:08.529236+0000","last_clean":"2024-03-21T08:55:08.529236+0000","last_became_active":"2024-03-21T08:55:07.532840+0000","last_became_peered":"2024-03-21T08:55:07.532840+0000","last_unstale":"2024-03-21T08:55:08.529236+0000","last_undegraded":"2024-03-21T08:55:08.529236+0000","last_fullsized":"2024-03-21T08:55:08.529236+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:52:36.565830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549900+0000","last_change":"2024-03-21T08:55:07.549259+0000","last_active":"2024-03-21T08:55:07.549900+0000","last_peered":"2024-03-21T08:55:07.549900+0000","last_clean":"2024-03-21T08:55:07.549900+0000","last_became_active":"2024-03-21T08:55:07.548161+0000","last_became_peered":"2024-03-21T08:55:07.548161+0000","last_unstale":"2024-03-21T08:55:07.549900+0000","last_undegraded":"2024-03-21T08:55:07.549900+0000","last_fullsized":"2024-03-21T08:55:07.549900+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:47:15.235830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126408+0000","last_change":"2024-03-21T08:55:07.530950+0000","last_active":"2024-03-21T08:55:09.126408+0000","last_peered":"2024-03-21T08:55:09.126408+0000","last_clean":"2024-03-21T08:55:09.126408+0000","last_became_active":"2024-03-21T08:55:07.530582+0000","last_became_peered":"2024-03-21T08:55:07.530582+0000","last_unstale":"2024-03-21T08:55:09.126408+0000","last_undegraded":"2024-03-21T08:55:09.126408+0000","last_fullsized":"2024-03-21T08:55:09.126408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:12.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126210+0000","last_change":"2024-03-21T08:55:07.535591+0000","last_active":"2024-03-21T08:55:09.126210+0000","last_peered":"2024-03-21T08:55:09.126210+0000","last_clean":"2024-03-21T08:55:09.126210+0000","last_became_active":"2024-03-21T08:55:07.535257+0000","last_became_peered":"2024-03-21T08:55:07.535257+0000","last_unstale":"2024-03-21T08:55:09.126210+0000","last_undegraded":"2024-03-21T08:55:09.126210+0000","last_fullsized":"2024-03-21T08:55:09.126210+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:39.383653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945766+0000","last_change":"2024-03-21T08:55:07.532165+0000","last_active":"2024-03-21T08:55:08.945766+0000","last_peered":"2024-03-21T08:55:08.945766+0000","last_clean":"2024-03-21T08:55:08.945766+0000","last_became_active":"2024-03-21T08:55:07.531822+0000","last_became_peered":"2024-03-21T08:55:07.531822+0000","last_unstale":"2024-03-21T08:55:08.945766+0000","last_undegraded":"2024-03-21T08:55:08.945766+0000","last_fullsized":"2024-03-21T08:55:08.945766+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:56.245424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.528055+0000","last_change":"2024-03-21T08:55:07.526371+0000","last_active":"2024-03-21T08:55:07.528055+0000","last_peered":"2024-03-21T08:55:07.528055+0000","last_clean":"2024-03-21T08:55:07.528055+0000","last_became_active":"2024-03-21T08:55:07.526021+0000","last_became_peered":"2024-03-21T08:55:07.526021+0000","last_unstale":"2024-03-21T08:55:07.528055+0000","last_undegraded":"2024-03-21T08:55:07.528055+0000","last_fullsized":"2024-03-21T08:55:07.528055+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:55:10.552953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126764+0000","last_change":"2024-03-21T08:55:07.532227+0000","last_active":"2024-03-21T08:55:09.126764+0000","last_peered":"2024-03-21T08:55:09.126764+0000","last_clean":"2024-03-21T08:55:09.126764+0000","last_became_active":"2024-03-21T08:55:07.531349+0000","last_became_peered":"2024-03-21T08:55:07.531349+0000","last_unstale":"2024-03-21T08:55:09.126764+0000","last_undegraded":"2024-03-21T08:55:09.126764+0000","last_fullsized":"2024-03-21T08:55:09.126764+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:07:40.318544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126857+0000","last_change":"2024-03-21T08:55:07.533531+0000","last_active":"2024-03-21T08:55:09.126857+0000","last_peered":"2024-03-21T08:55:09.126857+0000","last_clean":"2024-03-21T08:55:09.126857+0000","last_became_active":"2024-03-21T08:55:07.533166+0000","last_became_peered":"2024-03-21T08:55:07.533166+0000","last_unstale":"2024-03-21T08:55:09.126857+0000","last_undegraded":"2024-03-21T08:55:09.126857+0000","last_fullsized":"2024-03-21T08:55:09.126857+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:19:24.912748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529153+0000","last_change":"2024-03-21T08:55:07.533643+0000","last_active":"2024-03-21T08:55:08.529153+0000","last_peered":"2024-03-21T08:55:08.529153+0000","last_clean":"2024-03-21T08:55:08.529153+0000","last_became_active":"2024-03-21T08:55:07.533262+0000","last_became_peered":"2024-03-21T08:55:07.533262+0000","last_unstale":"2024-03-21T08:55:08.529153+0000","last_undegraded":"2024-03-21T08:55:08.529153+0000","last_fullsized":"2024-03-21T08:55:08.529153+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:07.641225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126326+0000","last_change":"2024-03-21T08:55:07.524670+0000","last_active":"2024-03-21T08:55:09.126326+0000","last_peered":"2024-03-21T08:55:09.126326+0000","last_clean":"2024-03-21T08:55:09.126326+0000","last_became_active":"2024-03-21T08:55:07.524295+0000","last_became_peered":"2024-03-21T08:55:07.524295+0000","last_unstale":"2024-03-21T08:55:09.126326+0000","last_undegraded":"2024-03-21T08:55:09.126326+0000","last_fullsized":"2024-03-21T08:55:09.126326+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:07.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.540004+0000","last_change":"2024-03-21T08:55:07.534401+0000","last_active":"2024-03-21T08:55:07.540004+0000","last_peered":"2024-03-21T08:55:07.540004+0000","last_clean":"2024-03-21T08:55:07.540004+0000","last_became_active":"2024-03-21T08:55:07.534056+0000","last_became_peered":"2024-03-21T08:55:07.534056+0000","last_unstale":"2024-03-21T08:55:07.540004+0000","last_undegraded":"2024-03-21T08:55:07.540004+0000","last_fullsized":"2024-03-21T08:55:07.540004+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:49:13.741149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.539847+0000","last_change":"2024-03-21T08:55:07.535915+0000","last_active":"2024-03-21T08:55:07.539847+0000","last_peered":"2024-03-21T08:55:07.539847+0000","last_clean":"2024-03-21T08:55:07.539847+0000","last_became_active":"2024-03-21T08:55:07.535540+0000","last_became_peered":"2024-03-21T08:55:07.535540+0000","last_unstale":"2024-03-21T08:55:07.539847+0000","last_undegraded":"2024-03-21T08:55:07.539847+0000","last_fullsized":"2024-03-21T08:55:07.539847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:45:02.189358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.543193+0000","last_change":"2024-03-21T08:55:07.541894+0000","last_active":"2024-03-21T08:55:07.543193+0000","last_peered":"2024-03-21T08:55:07.543193+0000","last_clean":"2024-03-21T08:55:07.543193+0000","last_became_active":"2024-03-21T08:55:07.541545+0000","last_became_peered":"2024-03-21T08:55:07.541545+0000","last_unstale":"2024-03-21T08:55:07.543193+0000","last_undegraded":"2024-03-21T08:55:07.543193+0000","last_fullsized":"2024-03-21T08:55:07.543193+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:22:41.033026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286370+0000","last_change":"2024-03-21T08:55:07.541228+0000","last_active":"2024-03-21T08:55:09.286370+0000","last_peered":"2024-03-21T08:55:09.286370+0000","last_clean":"2024-03-21T08:55:09.286370+0000","last_became_active":"2024-03-21T08:55:07.540836+0000","last_became_peered":"2024-03-21T08:55:07.540836+0000","last_unstale":"2024-03-21T08:55:09.286370+0000","last_undegraded":"2024-03-21T08:55:09.286370+0000","last_fullsized":"2024-03-21T08:55:09.286370+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:34:09.491557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530699+0000","last_change":"2024-03-21T08:55:07.525778+0000","last_active":"2024-03-21T08:55:08.530699+0000","last_peered":"2024-03-21T08:55:08.530699+0000","last_clean":"2024-03-21T08:55:08.530699+0000","last_became_active":"2024-03-21T08:55:07.525469+0000","last_became_peered":"2024-03-21T08:55:07.525469+0000","last_unstale":"2024-03-21T08:55:08.530699+0000","last_undegraded":"2024-03-21T08:55:08.530699+0000","last_fullsized":"2024-03-21T08:55:08.530699+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:41:31.715570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529315+0000","last_change":"2024-03-21T08:55:07.529929+0000","last_active":"2024-03-21T08:55:08.529315+0000","last_peered":"2024-03-21T08:55:08.529315+0000","last_clean":"2024-03-21T08:55:08.529315+0000","last_became_active":"2024-03-21T08:55:07.529615+0000","last_became_peered":"2024-03-21T08:55:07.529615+0000","last_unstale":"2024-03-21T08:55:08.529315+0000","last_undegraded":"2024-03-21T08:55:08.529315+0000","last_fullsized":"2024-03-21T08:55:08.529315+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:42:19.543201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530482+0000","last_change":"2024-03-21T08:55:06.239149+0000","last_active":"2024-03-21T08:55:08.530482+0000","last_peered":"2024-03-21T08:55:08.530482+0000","last_clean":"2024-03-21T08:55:08.530482+0000","last_became_active":"2024-03-21T08:55:06.238833+0000","last_became_peered":"2024-03-21T08:55:06.238833+0000","last_unstale":"2024-03-21T08:55:08.530482+0000","last_undegraded":"2024-03-21T08:55:08.530482+0000","last_fullsized":"2024-03-21T08:55:08.530482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:28.779442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946214+0000","last_change":"2024-03-21T08:55:07.533596+0000","last_active":"2024-03-21T08:55:08.946214+0000","last_peered":"2024-03-21T08:55:08.946214+0000","last_clean":"2024-03-21T08:55:08.946214+0000","last_became_active":"2024-03-21T08:55:07.533249+0000","last_became_peered":"2024-03-21T08:55:07.533249+0000","last_unstale":"2024-03-21T08:55:08.946214+0000","last_undegraded":"2024-03-21T08:55:08.946214+0000","last_fullsized":"2024-03-21T08:55:08.946214+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:21:40.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946109+0000","last_change":"2024-03-21T08:55:06.227045+0000","last_active":"2024-03-21T08:55:08.946109+0000","last_peered":"2024-03-21T08:55:08.946109+0000","last_clean":"2024-03-21T08:55:08.946109+0000","last_became_active":"2024-03-21T08:55:06.226748+0000","last_became_peered":"2024-03-21T08:55:06.226748+0000","last_unstale":"2024-03-21T08:55:08.946109+0000","last_undegraded":"2024-03-21T08:55:08.946109+0000","last_fullsized":"2024-03-21T08:55:08.946109+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:37.677094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.541147+0000","last_change":"2024-03-21T08:55:07.540024+0000","last_active":"2024-03-21T08:55:07.541147+0000","last_peered":"2024-03-21T08:55:07.541147+0000","last_clean":"2024-03-21T08:55:07.541147+0000","last_became_active":"2024-03-21T08:55:07.537574+0000","last_became_peered":"2024-03-21T08:55:07.537574+0000","last_unstale":"2024-03-21T08:55:07.541147+0000","last_undegraded":"2024-03-21T08:55:07.541147+0000","last_fullsized":"2024-03-21T08:55:07.541147+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:28.732768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.550396+0000","last_change":"2024-03-21T08:55:06.237261+0000","last_active":"2024-03-21T08:55:07.550396+0000","last_peered":"2024-03-21T08:55:07.550396+0000","last_clean":"2024-03-21T08:55:07.550396+0000","last_became_active":"2024-03-21T08:55:06.236838+0000","last_became_peered":"2024-03-21T08:55:06.236838+0000","last_unstale":"2024-03-21T08:55:07.550396+0000","last_undegraded":"2024-03-21T08:55:07.550396+0000","last_fullsized":"2024-03-21T08:55:07.550396+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:59.881075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.530299+0000","last_change":"2024-03-21T08:55:07.529159+0000","last_active":"2024-03-21T08:55:07.530299+0000","last_peered":"2024-03-21T08:55:07.530299+0000","last_clean":"2024-03-21T08:55:07.530299+0000","last_became_active":"2024-03-21T08:55:07.527260+0000","last_became_peered":"2024-03-21T08:55:07.527260+0000","last_unstale":"2024-03-21T08:55:07.530299+0000","last_undegraded":"2024-03-21T08:55:07.530299+0000","last_fullsized":"2024-03-21T08:55:07.530299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:00:30.020729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549501+0000","last_change":"2024-03-21T08:55:06.232959+0000","last_active":"2024-03-21T08:55:07.549501+0000","last_peered":"2024-03-21T08:55:07.549501+0000","last_clean":"2024-03-21T08:55:07.549501+0000","last_became_active":"2024-03-21T08:55:06.232606+0000","last_became_peered":"2024-03-21T08:55:06.232606+0000","last_unstale":"2024-03-21T08:55:07.549501+0000","last_undegraded":"2024-03-21T08:55:07.549501+0000","last_fullsized":"2024-03-21T08:55:07.549501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:27:07.480767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126725+0000","last_change":"2024-03-21T08:55:07.530711+0000","last_active":"2024-03-21T08:55:09.126725+0000","last_peered":"2024-03-21T08:55:09.126725+0000","last_clean":"2024-03-21T08:55:09.126725+0000","last_became_active":"2024-03-21T08:55:07.530374+0000","last_became_peered":"2024-03-21T08:55:07.530374+0000","last_unstale":"2024-03-21T08:55:09.126725+0000","last_undegraded":"2024-03-21T08:55:09.126725+0000","last_fullsized":"2024-03-21T08:55:09.126725+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:06.524172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.941262+0000","last_change":"2024-03-21T08:55:06.233934+0000","last_active":"2024-03-21T08:55:09.941262+0000","last_peered":"2024-03-21T08:55:09.941262+0000","last_clean":"2024-03-21T08:55:09.941262+0000","last_became_active":"2024-03-21T08:55:06.233618+0000","last_became_peered":"2024-03-21T08:55:06.233618+0000","last_unstale":"2024-03-21T08:55:09.941262+0000","last_undegraded":"2024-03-21T08:55:09.941262+0000","last_fullsized":"2024-03-21T08:55:09.941262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:08.769883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.548183+0000","last_change":"2024-03-21T08:55:07.546411+0000","last_active":"2024-03-21T08:55:07.548183+0000","last_peered":"2024-03-21T08:55:07.548183+0000","last_clean":"2024-03-21T08:55:07.548183+0000","last_became_active":"2024-03-21T08:55:07.546051+0000","last_became_peered":"2024-03-21T08:55:07.546051+0000","last_unstale":"2024-03-21T08:55:07.548183+0000","last_undegraded":"2024-03-21T08:55:07.548183+0000","last_fullsized":"2024-03-21T08:55:07.548183+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:54:28.820026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126250+0000","last_change":"2024-03-21T08:55:06.226791+0000","last_active":"2024-03-21T08:55:09.126250+0000","last_peered":"2024-03-21T08:55:09.126250+0000","last_clean":"2024-03-21T08:55:09.126250+0000","last_became_active":"2024-03-21T08:55:06.226438+0000","last_became_peered":"2024-03-21T08:55:06.226438+0000","last_unstale":"2024-03-21T08:55:09.126250+0000","last_undegraded":"2024-03-21T08:55:09.126250+0000","last_fullsized":"2024-03-21T08:55:09.126250+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:15:03.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126881+0000","last_change":"2024-03-21T08:55:07.529623+0000","last_active":"2024-03-21T08:55:09.126881+0000","last_peered":"2024-03-21T08:55:09.126881+0000","last_clean":"2024-03-21T08:55:09.126881+0000","last_became_active":"2024-03-21T08:55:07.529293+0000","last_became_peered":"2024-03-21T08:55:07.529293+0000","last_unstale":"2024-03-21T08:55:09.126881+0000","last_undegraded":"2024-03-21T08:55:09.126881+0000","last_fullsized":"2024-03-21T08:55:09.126881+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:43.265861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.944837+0000","last_change":"2024-03-21T08:55:06.236381+0000","last_active":"2024-03-21T08:55:09.944837+0000","last_peered":"2024-03-21T08:55:09.944837+0000","last_clean":"2024-03-21T08:55:09.944837+0000","last_became_active":"2024-03-21T08:55:06.236071+0000","last_became_peered":"2024-03-21T08:55:06.236071+0000","last_unstale":"2024-03-21T08:55:09.944837+0000","last_undegraded":"2024-03-21T08:55:09.944837+0000","last_fullsized":"2024-03-21T08:55:09.944837+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:13.338191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945840+0000","last_change":"2024-03-21T08:55:07.529637+0000","last_active":"2024-03-21T08:55:08.945840+0000","last_peered":"2024-03-21T08:55:08.945840+0000","last_clean":"2024-03-21T08:55:08.945840+0000","last_became_active":"2024-03-21T08:55:07.527536+0000","last_became_peered":"2024-03-21T08:55:07.527536+0000","last_unstale":"2024-03-21T08:55:08.945840+0000","last_undegraded":"2024-03-21T08:55:08.945840+0000","last_fullsized":"2024-03-21T08:55:08.945840+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:15:58.623055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"86'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.553938+0000","last_change":"2024-03-21T08:55:06.233121+0000","last_active":"2024-03-21T08:55:09.553938+0000","last_peered":"2024-03-21T08:55:09.553938+0000","last_clean":"2024-03-21T08:55:09.553938+0000","last_became_active":"2024-03-21T08:55:06.232772+0000","last_became_peered":"2024-03-21T08:55:06.232772+0000","last_unstale":"2024-03-21T08:55:09.553938+0000","last_undegraded":"2024-03-21T08:55:09.553938+0000","last_fullsized":"2024-03-21T08:55:09.553938+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:05.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530394+0000","last_change":"2024-03-21T08:55:07.527483+0000","last_active":"2024-03-21T08:55:08.530394+0000","last_peered":"2024-03-21T08:55:08.530394+0000","last_clean":"2024-03-21T08:55:08.530394+0000","last_became_active":"2024-03-21T08:55:07.527038+0000","last_became_peered":"2024-03-21T08:55:07.527038+0000","last_unstale":"2024-03-21T08:55:08.530394+0000","last_undegraded":"2024-03-21T08:55:08.530394+0000","last_fullsized":"2024-03-21T08:55:08.530394+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:27:13.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.554813+0000","last_change":"2024-03-21T08:55:06.233677+0000","last_active":"2024-03-21T08:55:09.554813+0000","last_peered":"2024-03-21T08:55:09.554813+0000","last_clean":"2024-03-21T08:55:09.554813+0000","last_became_active":"2024-03-21T08:55:06.233312+0000","last_became_peered":"2024-03-21T08:55:06.233312+0000","last_unstale":"2024-03-21T08:55:09.554813+0000","last_undegraded":"2024-03-21T08:55:09.554813+0000","last_fullsized":"2024-03-21T08:55:09.554813+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:51:53.894648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.529152+0000","last_change":"2024-03-21T08:55:07.528174+0000","last_active":"2024-03-21T08:55:07.529152+0000","last_peered":"2024-03-21T08:55:07.529152+0000","last_clean":"2024-03-21T08:55:07.529152+0000","last_became_active":"2024-03-21T08:55:07.527313+0000","last_became_peered":"2024-03-21T08:55:07.527313+0000","last_unstale":"2024-03-21T08:55:07.529152+0000","last_undegraded":"2024-03-21T08:55:07.529152+0000","last_fullsized":"2024-03-21T08:55:07.529152+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:53:47.876242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.962858+0000","last_change":"2024-03-21T08:55:06.237324+0000","last_active":"2024-03-21T08:55:11.962858+0000","last_peered":"2024-03-21T08:55:11.962858+0000","last_clean":"2024-03-21T08:55:11.962858+0000","last_became_active":"2024-03-21T08:55:06.235319+0000","last_became_peered":"2024-03-21T08:55:06.235319+0000","last_unstale":"2024-03-21T08:55:11.962858+0000","last_undegraded":"2024-03-21T08:55:11.962858+0000","last_fullsized":"2024-03-21T08:55:11.962858+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:05:32.872612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.544820+0000","last_change":"2024-03-21T08:55:07.540552+0000","last_active":"2024-03-21T08:55:07.544820+0000","last_peered":"2024-03-21T08:55:07.544820+0000","last_clean":"2024-03-21T08:55:07.544820+0000","last_became_active":"2024-03-21T08:55:07.540211+0000","last_became_peered":"2024-03-21T08:55:07.540211+0000","last_unstale":"2024-03-21T08:55:07.544820+0000","last_undegraded":"2024-03-21T08:55:07.544820+0000","last_fullsized":"2024-03-21T08:55:07.544820+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:00:46.298334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.940125+0000","last_change":"2024-03-21T08:55:06.228420+0000","last_active":"2024-03-21T08:55:09.940125+0000","last_peered":"2024-03-21T08:55:09.940125+0000","last_clean":"2024-03-21T08:55:09.940125+0000","last_became_active":"2024-03-21T08:55:06.228037+0000","last_became_peered":"2024-03-21T08:55:06.228037+0000","last_unstale":"2024-03-21T08:55:09.940125+0000","last_undegraded":"2024-03-21T08:55:09.940125+0000","last_fullsized":"2024-03-21T08:55:09.940125+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:22.437321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.534498+0000","last_change":"2024-03-21T08:55:07.534053+0000","last_active":"2024-03-21T08:55:07.534498+0000","last_peered":"2024-03-21T08:55:07.534498+0000","last_clean":"2024-03-21T08:55:07.534498+0000","last_became_active":"2024-03-21T08:55:07.533404+0000","last_became_peered":"2024-03-21T08:55:07.533404+0000","last_unstale":"2024-03-21T08:55:07.534498+0000","last_undegraded":"2024-03-21T08:55:07.534498+0000","last_fullsized":"2024-03-21T08:55:07.534498+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:09:11.414407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.554423+0000","last_change":"2024-03-21T08:55:06.232681+0000","last_active":"2024-03-21T08:55:09.554423+0000","last_peered":"2024-03-21T08:55:09.554423+0000","last_clean":"2024-03-21T08:55:09.554423+0000","last_became_active":"2024-03-21T08:55:06.232109+0000","last_became_peered":"2024-03-21T08:55:06.232109+0000","last_unstale":"2024-03-21T08:55:09.554423+0000","last_undegraded":"2024-03-21T08:55:09.554423+0000","last_fullsized":"2024-03-21T08:55:09.554423+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:11.218106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549378+0000","last_change":"2024-03-21T08:55:07.546464+0000","last_active":"2024-03-21T08:55:07.549378+0000","last_peered":"2024-03-21T08:55:07.549378+0000","last_clean":"2024-03-21T08:55:07.549378+0000","last_became_active":"2024-03-21T08:55:07.546111+0000","last_became_peered":"2024-03-21T08:55:07.546111+0000","last_unstale":"2024-03-21T08:55:07.549378+0000","last_undegraded":"2024-03-21T08:55:07.549378+0000","last_fullsized":"2024-03-21T08:55:07.549378+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:03:12.732453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":25,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.958692+0000","last_change":"2024-03-21T08:55:06.233157+0000","last_active":"2024-03-21T08:55:11.958692+0000","last_peered":"2024-03-21T08:55:11.958692+0000","last_clean":"2024-03-21T08:55:11.958692+0000","last_became_active":"2024-03-21T08:55:06.232302+0000","last_became_peered":"2024-03-21T08:55:06.232302+0000","last_unstale":"2024-03-21T08:55:11.958692+0000","last_undegraded":"2024-03-21T08:55:11.958692+0000","last_fullsized":"2024-03-21T08:55:11.958692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:47.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126048+0000","last_change":"2024-03-21T08:55:07.531265+0000","last_active":"2024-03-21T08:55:09.126048+0000","last_peered":"2024-03-21T08:55:09.126048+0000","last_clean":"2024-03-21T08:55:09.126048+0000","last_became_active":"2024-03-21T08:55:07.530913+0000","last_became_peered":"2024-03-21T08:55:07.530913+0000","last_unstale":"2024-03-21T08:55:09.126048+0000","last_undegraded":"2024-03-21T08:55:09.126048+0000","last_fullsized":"2024-03-21T08:55:09.126048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:34.171463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.553830+0000","last_change":"2024-03-21T08:55:06.233411+0000","last_active":"2024-03-21T08:55:09.553830+0000","last_peered":"2024-03-21T08:55:09.553830+0000","last_clean":"2024-03-21T08:55:09.553830+0000","last_became_active":"2024-03-21T08:55:06.233031+0000","last_became_peered":"2024-03-21T08:55:06.233031+0000","last_unstale":"2024-03-21T08:55:09.553830+0000","last_undegraded":"2024-03-21T08:55:09.553830+0000","last_fullsized":"2024-03-21T08:55:09.553830+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:52:07.258184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126818+0000","last_change":"2024-03-21T08:55:07.531557+0000","last_active":"2024-03-21T08:55:09.126818+0000","last_peered":"2024-03-21T08:55:09.126818+0000","last_clean":"2024-03-21T08:55:09.126818+0000","last_became_active":"2024-03-21T08:55:07.531183+0000","last_became_peered":"2024-03-21T08:55:07.531183+0000","last_unstale":"2024-03-21T08:55:09.126818+0000","last_undegraded":"2024-03-21T08:55:09.126818+0000","last_fullsized":"2024-03-21T08:55:09.126818+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:14:15.401964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.951629+0000","last_change":"2024-03-21T08:55:06.230587+0000","last_active":"2024-03-21T08:55:10.951629+0000","last_peered":"2024-03-21T08:55:10.951629+0000","last_clean":"2024-03-21T08:55:10.951629+0000","last_became_active":"2024-03-21T08:55:06.230201+0000","last_became_peered":"2024-03-21T08:55:06.230201+0000","last_unstale":"2024-03-21T08:55:10.951629+0000","last_undegraded":"2024-03-21T08:55:10.951629+0000","last_fullsized":"2024-03-21T08:55:10.951629+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:01.954850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126330+0000","last_change":"2024-03-21T08:55:07.532592+0000","last_active":"2024-03-21T08:55:09.126330+0000","last_peered":"2024-03-21T08:55:09.126330+0000","last_clean":"2024-03-21T08:55:09.126330+0000","last_became_active":"2024-03-21T08:55:07.532210+0000","last_became_peered":"2024-03-21T08:55:07.532210+0000","last_unstale":"2024-03-21T08:55:09.126330+0000","last_undegraded":"2024-03-21T08:55:09.126330+0000","last_fullsized":"2024-03-21T08:55:09.126330+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:59.775085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'8","reported_seq":21,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.550788+0000","last_change":"2024-03-21T08:55:06.229096+0000","last_active":"2024-03-21T08:55:07.550788+0000","last_peered":"2024-03-21T08:55:07.550788+0000","last_clean":"2024-03-21T08:55:07.550788+0000","last_became_active":"2024-03-21T08:55:06.228772+0000","last_became_peered":"2024-03-21T08:55:06.228772+0000","last_unstale":"2024-03-21T08:55:07.550788+0000","last_undegraded":"2024-03-21T08:55:07.550788+0000","last_fullsized":"2024-03-21T08:55:07.550788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:05:31.017977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126275+0000","last_change":"2024-03-21T08:55:07.527968+0000","last_active":"2024-03-21T08:55:09.126275+0000","last_peered":"2024-03-21T08:55:09.126275+0000","last_clean":"2024-03-21T08:55:09.126275+0000","last_became_active":"2024-03-21T08:55:07.523326+0000","last_became_peered":"2024-03-21T08:55:07.523326+0000","last_unstale":"2024-03-21T08:55:09.126275+0000","last_undegraded":"2024-03-21T08:55:09.126275+0000","last_fullsized":"2024-03-21T08:55:09.126275+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:01:22.903576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'1","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.549492+0000","last_change":"2024-03-21T08:55:06.225599+0000","last_active":"2024-03-21T08:55:07.549492+0000","last_peered":"2024-03-21T08:55:07.549492+0000","last_clean":"2024-03-21T08:55:07.549492+0000","last_became_active":"2024-03-21T08:55:06.225205+0000","last_became_peered":"2024-03-21T08:55:06.225205+0000","last_unstale":"2024-03-21T08:55:07.549492+0000","last_undegraded":"2024-03-21T08:55:07.549492+0000","last_fullsized":"2024-03-21T08:55:07.549492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:23:07.286408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286116+0000","last_change":"2024-03-21T08:55:07.532577+0000","last_active":"2024-03-21T08:55:09.286116+0000","last_peered":"2024-03-21T08:55:09.286116+0000","last_clean":"2024-03-21T08:55:09.286116+0000","last_became_active":"2024-03-21T08:55:07.529015+0000","last_became_peered":"2024-03-21T08:55:07.529015+0000","last_unstale":"2024-03-21T08:55:09.286116+0000","last_undegraded":"2024-03-21T08:55:09.286116+0000","last_fullsized":"2024-03-21T08:55:09.286116+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:24.045732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530544+0000","last_change":"2024-03-21T08:55:06.238430+0000","last_active":"2024-03-21T08:55:08.530544+0000","last_peered":"2024-03-21T08:55:08.530544+0000","last_clean":"2024-03-21T08:55:08.530544+0000","last_became_active":"2024-03-21T08:55:06.238128+0000","last_became_peered":"2024-03-21T08:55:06.238128+0000","last_unstale":"2024-03-21T08:55:08.530544+0000","last_undegraded":"2024-03-21T08:55:08.530544+0000","last_fullsized":"2024-03-21T08:55:08.530544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:49.839546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126810+0000","last_change":"2024-03-21T08:55:07.540750+0000","last_active":"2024-03-21T08:55:09.126810+0000","last_peered":"2024-03-21T08:55:09.126810+0000","last_clean":"2024-03-21T08:55:09.126810+0000","last_became_active":"2024-03-21T08:55:07.540358+0000","last_became_peered":"2024-03-21T08:55:07.540358+0000","last_unstale":"2024-03-21T08:55:09.126810+0000","last_undegraded":"2024-03-21T08:55:09.126810+0000","last_fullsized":"2024-03-21T08:55:09.126810+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:37.590268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.963020+0000","last_change":"2024-03-21T08:55:06.236387+0000","last_active":"2024-03-21T08:55:11.963020+0000","last_peered":"2024-03-21T08:55:11.963020+0000","last_clean":"2024-03-21T08:55:11.963020+0000","last_became_active":"2024-03-21T08:55:06.234603+0000","last_became_peered":"2024-03-21T08:55:06.234603+0000","last_unstale":"2024-03-21T08:55:11.963020+0000","last_undegraded":"2024-03-21T08:55:11.963020+0000","last_fullsized":"2024-03-21T08:55:11.963020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:10:28.921679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126959+0000","last_change":"2024-03-21T08:55:07.539011+0000","last_active":"2024-03-21T08:55:09.126959+0000","last_peered":"2024-03-21T08:55:09.126959+0000","last_clean":"2024-03-21T08:55:09.126959+0000","last_became_active":"2024-03-21T08:55:07.538694+0000","last_became_peered":"2024-03-21T08:55:07.538694+0000","last_unstale":"2024-03-21T08:55:09.126959+0000","last_undegraded":"2024-03-21T08:55:09.126959+0000","last_fullsized":"2024-03-21T08:55:09.126959+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:52.746409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126303+0000","last_change":"2024-03-21T08:55:07.531605+0000","last_active":"2024-03-21T08:55:09.126303+0000","last_peered":"2024-03-21T08:55:09.126303+0000","last_clean":"2024-03-21T08:55:09.126303+0000","last_became_active":"2024-03-21T08:55:07.530207+0000","last_became_peered":"2024-03-21T08:55:07.530207+0000","last_unstale":"2024-03-21T08:55:09.126303+0000","last_undegraded":"2024-03-21T08:55:09.126303+0000","last_fullsized":"2024-03-21T08:55:09.126303+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:58.805184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.945664+0000","last_change":"2024-03-21T08:55:06.228108+0000","last_active":"2024-03-21T08:55:09.945664+0000","last_peered":"2024-03-21T08:55:09.945664+0000","last_clean":"2024-03-21T08:55:09.945664+0000","last_became_active":"2024-03-21T08:55:06.227281+0000","last_became_peered":"2024-03-21T08:55:06.227281+0000","last_unstale":"2024-03-21T08:55:09.945664+0000","last_undegraded":"2024-03-21T08:55:09.945664+0000","last_fullsized":"2024-03-21T08:55:09.945664+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:02.026031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946052+0000","last_change":"2024-03-21T08:55:07.531090+0000","last_active":"2024-03-21T08:55:08.946052+0000","last_peered":"2024-03-21T08:55:08.946052+0000","last_clean":"2024-03-21T08:55:08.946052+0000","last_became_active":"2024-03-21T08:55:07.530769+0000","last_became_peered":"2024-03-21T08:55:07.530769+0000","last_unstale":"2024-03-21T08:55:08.946052+0000","last_undegraded":"2024-03-21T08:55:08.946052+0000","last_fullsized":"2024-03-21T08:55:08.946052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:49:51.950743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.953349+0000","last_change":"2024-03-21T08:55:06.228583+0000","last_active":"2024-03-21T08:55:10.953349+0000","last_peered":"2024-03-21T08:55:10.953349+0000","last_clean":"2024-03-21T08:55:10.953349+0000","last_became_active":"2024-03-21T08:55:06.228222+0000","last_became_peered":"2024-03-21T08:55:06.228222+0000","last_unstale":"2024-03-21T08:55:10.953349+0000","last_undegraded":"2024-03-21T08:55:10.953349+0000","last_fullsized":"2024-03-21T08:55:10.953349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:13:28.254775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":340,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126326+0000","last_change":"2024-03-21T08:53:52.956820+0000","last_active":"2024-03-21T08:55:09.126326+0000","last_peered":"2024-03-21T08:55:09.126326+0000","last_clean":"2024-03-21T08:55:09.126326+0000","last_became_active":"2024-03-21T08:53:52.956322+0000","last_became_peered":"2024-03-21T08:53:52.956322+0000","last_unstale":"2024-03-21T08:55:09.126326+0000","last_undegraded":"2024-03-21T08:55:09.126326+0000","last_fullsized":"2024-03-21T08:55:09.126326+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_clean_scrub_stamp":"2024-03-21T08:50:27.622697+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:55.824313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.125902+0000","last_change":"2024-03-21T08:55:07.529927+0000","last_active":"2024-03-21T08:55:09.125902+0000","last_peered":"2024-03-21T08:55:09.125902+0000","last_clean":"2024-03-21T08:55:09.125902+0000","last_became_active":"2024-03-21T08:55:07.529588+0000","last_became_peered":"2024-03-21T08:55:07.529588+0000","last_unstale":"2024-03-21T08:55:09.125902+0000","last_undegraded":"2024-03-21T08:55:09.125902+0000","last_fullsized":"2024-03-21T08:55:09.125902+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:11:04.122448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.945339+0000","last_change":"2024-03-21T08:55:06.236496+0000","last_active":"2024-03-21T08:55:09.945339+0000","last_peered":"2024-03-21T08:55:09.945339+0000","last_clean":"2024-03-21T08:55:09.945339+0000","last_became_active":"2024-03-21T08:55:06.234447+0000","last_became_peered":"2024-03-21T08:55:06.234447+0000","last_unstale":"2024-03-21T08:55:09.945339+0000","last_undegraded":"2024-03-21T08:55:09.945339+0000","last_fullsized":"2024-03-21T08:55:09.945339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:46:55.322330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.523352+0000","last_change":"2024-03-21T08:55:06.232367+0000","last_active":"2024-03-21T08:55:07.523352+0000","last_peered":"2024-03-21T08:55:07.523352+0000","last_clean":"2024-03-21T08:55:07.523352+0000","last_became_active":"2024-03-21T08:55:06.229741+0000","last_became_peered":"2024-03-21T08:55:06.229741+0000","last_unstale":"2024-03-21T08:55:07.523352+0000","last_undegraded":"2024-03-21T08:55:07.523352+0000","last_fullsized":"2024-03-21T08:55:07.523352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:51.644926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126261+0000","last_change":"2024-03-21T08:55:07.527366+0000","last_active":"2024-03-21T08:55:09.126261+0000","last_peered":"2024-03-21T08:55:09.126261+0000","last_clean":"2024-03-21T08:55:09.126261+0000","last_became_active":"2024-03-21T08:55:07.527033+0000","last_became_peered":"2024-03-21T08:55:07.527033+0000","last_unstale":"2024-03-21T08:55:09.126261+0000","last_undegraded":"2024-03-21T08:55:09.126261+0000","last_fullsized":"2024-03-21T08:55:09.126261+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:07:38.400820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.530615+0000","last_change":"2024-03-21T08:55:06.234078+0000","last_active":"2024-03-21T08:55:08.530615+0000","last_peered":"2024-03-21T08:55:08.530615+0000","last_clean":"2024-03-21T08:55:08.530615+0000","last_became_active":"2024-03-21T08:55:06.233704+0000","last_became_peered":"2024-03-21T08:55:06.233704+0000","last_unstale":"2024-03-21T08:55:08.530615+0000","last_undegraded":"2024-03-21T08:55:08.530615+0000","last_fullsized":"2024-03-21T08:55:08.530615+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:08:00.957142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946235+0000","last_change":"2024-03-21T08:55:07.529668+0000","last_active":"2024-03-21T08:55:08.946235+0000","last_peered":"2024-03-21T08:55:08.946235+0000","last_clean":"2024-03-21T08:55:08.946235+0000","last_became_active":"2024-03-21T08:55:07.528560+0000","last_became_peered":"2024-03-21T08:55:07.528560+0000","last_unstale":"2024-03-21T08:55:08.946235+0000","last_undegraded":"2024-03-21T08:55:08.946235+0000","last_fullsized":"2024-03-21T08:55:08.946235+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:49.671107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":22,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.961748+0000","last_change":"2024-03-21T08:55:06.241169+0000","last_active":"2024-03-21T08:55:11.961748+0000","last_peered":"2024-03-21T08:55:11.961748+0000","last_clean":"2024-03-21T08:55:11.961748+0000","last_became_active":"2024-03-21T08:55:06.240740+0000","last_became_peered":"2024-03-21T08:55:06.240740+0000","last_unstale":"2024-03-21T08:55:11.961748+0000","last_undegraded":"2024-03-21T08:55:11.961748+0000","last_fullsized":"2024-03-21T08:55:11.961748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:31.327983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.546595+0000","last_change":"2024-03-21T08:55:07.544243+0000","last_active":"2024-03-21T08:55:07.546595+0000","last_peered":"2024-03-21T08:55:07.546595+0000","last_clean":"2024-03-21T08:55:07.546595+0000","last_became_active":"2024-03-21T08:55:07.543827+0000","last_became_peered":"2024-03-21T08:55:07.543827+0000","last_unstale":"2024-03-21T08:55:07.546595+0000","last_undegraded":"2024-03-21T08:55:07.546595+0000","last_fullsized":"2024-03-21T08:55:07.546595+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:30:07.859809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.512267+0000","last_change":"2024-03-21T08:55:06.236769+0000","last_active":"2024-03-21T08:55:07.512267+0000","last_peered":"2024-03-21T08:55:07.512267+0000","last_clean":"2024-03-21T08:55:07.512267+0000","last_became_active":"2024-03-21T08:55:06.236426+0000","last_became_peered":"2024-03-21T08:55:06.236426+0000","last_unstale":"2024-03-21T08:55:07.512267+0000","last_undegraded":"2024-03-21T08:55:07.512267+0000","last_fullsized":"2024-03-21T08:55:07.512267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:28:51.833976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.545073+0000","last_change":"2024-03-21T08:55:07.540974+0000","last_active":"2024-03-21T08:55:07.545073+0000","last_peered":"2024-03-21T08:55:07.545073+0000","last_clean":"2024-03-21T08:55:07.545073+0000","last_became_active":"2024-03-21T08:55:07.540246+0000","last_became_peered":"2024-03-21T08:55:07.540246+0000","last_unstale":"2024-03-21T08:55:07.545073+0000","last_undegraded":"2024-03-21T08:55:07.545073+0000","last_fullsized":"2024-03-21T08:55:07.545073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:26:27.624121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.953272+0000","last_change":"2024-03-21T08:55:06.231544+0000","last_active":"2024-03-21T08:55:10.953272+0000","last_peered":"2024-03-21T08:55:10.953272+0000","last_clean":"2024-03-21T08:55:10.953272+0000","last_became_active":"2024-03-21T08:55:06.231238+0000","last_became_peered":"2024-03-21T08:55:06.231238+0000","last_unstale":"2024-03-21T08:55:10.953272+0000","last_undegraded":"2024-03-21T08:55:10.953272+0000","last_fullsized":"2024-03-21T08:55:10.953272+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:34.831596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.285957+0000","last_change":"2024-03-21T08:55:07.541189+0000","last_active":"2024-03-21T08:55:09.285957+0000","last_peered":"2024-03-21T08:55:09.285957+0000","last_clean":"2024-03-21T08:55:09.285957+0000","last_became_active":"2024-03-21T08:55:07.540819+0000","last_became_peered":"2024-03-21T08:55:07.540819+0000","last_unstale":"2024-03-21T08:55:09.285957+0000","last_undegraded":"2024-03-21T08:55:09.285957+0000","last_fullsized":"2024-03-21T08:55:09.285957+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:21:43.879895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.953117+0000","last_change":"2024-03-21T08:55:06.231164+0000","last_active":"2024-03-21T08:55:10.953117+0000","last_peered":"2024-03-21T08:55:10.953117+0000","last_clean":"2024-03-21T08:55:10.953117+0000","last_became_active":"2024-03-21T08:55:06.230782+0000","last_became_peered":"2024-03-21T08:55:06.230782+0000","last_unstale":"2024-03-21T08:55:10.953117+0000","last_undegraded":"2024-03-21T08:55:10.953117+0000","last_fullsized":"2024-03-21T08:55:10.953117+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:26:56.744957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945989+0000","last_change":"2024-03-21T08:55:07.544379+0000","last_active":"2024-03-21T08:55:08.945989+0000","last_peered":"2024-03-21T08:55:08.945989+0000","last_clean":"2024-03-21T08:55:08.945989+0000","last_became_active":"2024-03-21T08:55:07.541643+0000","last_became_peered":"2024-03-21T08:55:07.541643+0000","last_unstale":"2024-03-21T08:55:08.945989+0000","last_undegraded":"2024-03-21T08:55:08.945989+0000","last_fullsized":"2024-03-21T08:55:08.945989+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:54:19.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'2","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.550522+0000","last_change":"2024-03-21T08:55:06.229859+0000","last_active":"2024-03-21T08:55:07.550522+0000","last_peered":"2024-03-21T08:55:07.550522+0000","last_clean":"2024-03-21T08:55:07.550522+0000","last_became_active":"2024-03-21T08:55:06.229532+0000","last_became_peered":"2024-03-21T08:55:06.229532+0000","last_unstale":"2024-03-21T08:55:07.550522+0000","last_undegraded":"2024-03-21T08:55:07.550522+0000","last_fullsized":"2024-03-21T08:55:07.550522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:39.615417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126847+0000","last_change":"2024-03-21T08:55:07.530619+0000","last_active":"2024-03-21T08:55:09.126847+0000","last_peered":"2024-03-21T08:55:09.126847+0000","last_clean":"2024-03-21T08:55:09.126847+0000","last_became_active":"2024-03-21T08:55:07.530293+0000","last_became_peered":"2024-03-21T08:55:07.530293+0000","last_unstale":"2024-03-21T08:55:09.126847+0000","last_undegraded":"2024-03-21T08:55:09.126847+0000","last_fullsized":"2024-03-21T08:55:09.126847+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:44:56.242621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:10.951888+0000","last_change":"2024-03-21T08:55:06.241581+0000","last_active":"2024-03-21T08:55:10.951888+0000","last_peered":"2024-03-21T08:55:10.951888+0000","last_clean":"2024-03-21T08:55:10.951888+0000","last_became_active":"2024-03-21T08:55:06.241162+0000","last_became_peered":"2024-03-21T08:55:06.241162+0000","last_unstale":"2024-03-21T08:55:10.951888+0000","last_undegraded":"2024-03-21T08:55:10.951888+0000","last_fullsized":"2024-03-21T08:55:10.951888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:15.231840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.127064+0000","last_change":"2024-03-21T08:55:07.532424+0000","last_active":"2024-03-21T08:55:09.127064+0000","last_peered":"2024-03-21T08:55:09.127064+0000","last_clean":"2024-03-21T08:55:09.127064+0000","last_became_active":"2024-03-21T08:55:07.532062+0000","last_became_peered":"2024-03-21T08:55:07.532062+0000","last_unstale":"2024-03-21T08:55:09.127064+0000","last_undegraded":"2024-03-21T08:55:09.127064+0000","last_fullsized":"2024-03-21T08:55:09.127064+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:44:07.970387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'2","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.936262+0000","last_change":"2024-03-21T08:55:06.232191+0000","last_active":"2024-03-21T08:55:09.936262+0000","last_peered":"2024-03-21T08:55:09.936262+0000","last_clean":"2024-03-21T08:55:09.936262+0000","last_became_active":"2024-03-21T08:55:06.231880+0000","last_became_peered":"2024-03-21T08:55:06.231880+0000","last_unstale":"2024-03-21T08:55:09.936262+0000","last_undegraded":"2024-03-21T08:55:09.936262+0000","last_fullsized":"2024-03-21T08:55:09.936262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:35.434959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126310+0000","last_change":"2024-03-21T08:55:07.538588+0000","last_active":"2024-03-21T08:55:09.126310+0000","last_peered":"2024-03-21T08:55:09.126310+0000","last_clean":"2024-03-21T08:55:09.126310+0000","last_became_active":"2024-03-21T08:55:07.538268+0000","last_became_peered":"2024-03-21T08:55:07.538268+0000","last_unstale":"2024-03-21T08:55:09.126310+0000","last_undegraded":"2024-03-21T08:55:09.126310+0000","last_fullsized":"2024-03-21T08:55:09.126310+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:21.591094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":20,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:11.960871+0000","last_change":"2024-03-21T08:55:06.228166+0000","last_active":"2024-03-21T08:55:11.960871+0000","last_peered":"2024-03-21T08:55:11.960871+0000","last_clean":"2024-03-21T08:55:11.960871+0000","last_became_active":"2024-03-21T08:55:06.227852+0000","last_became_peered":"2024-03-21T08:55:06.227852+0000","last_unstale":"2024-03-21T08:55:11.960871+0000","last_undegraded":"2024-03-21T08:55:11.960871+0000","last_fullsized":"2024-03-21T08:55:11.960871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:12.471809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126978+0000","last_change":"2024-03-21T08:55:07.536854+0000","last_active":"2024-03-21T08:55:09.126978+0000","last_peered":"2024-03-21T08:55:09.126978+0000","last_clean":"2024-03-21T08:55:09.126978+0000","last_became_active":"2024-03-21T08:55:07.536241+0000","last_became_peered":"2024-03-21T08:55:07.536241+0000","last_unstale":"2024-03-21T08:55:09.126978+0000","last_undegraded":"2024-03-21T08:55:09.126978+0000","last_fullsized":"2024-03-21T08:55:09.126978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:21.060162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286151+0000","last_change":"2024-03-21T08:55:07.528084+0000","last_active":"2024-03-21T08:55:09.286151+0000","last_peered":"2024-03-21T08:55:09.286151+0000","last_clean":"2024-03-21T08:55:09.286151+0000","last_became_active":"2024-03-21T08:55:07.527675+0000","last_became_peered":"2024-03-21T08:55:07.527675+0000","last_unstale":"2024-03-21T08:55:09.286151+0000","last_undegraded":"2024-03-21T08:55:09.286151+0000","last_fullsized":"2024-03-21T08:55:09.286151+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:47:21.646078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529049+0000","last_change":"2024-03-21T08:55:07.542384+0000","last_active":"2024-03-21T08:55:08.529049+0000","last_peered":"2024-03-21T08:55:08.529049+0000","last_clean":"2024-03-21T08:55:08.529049+0000","last_became_active":"2024-03-21T08:55:07.541956+0000","last_became_peered":"2024-03-21T08:55:07.541956+0000","last_unstale":"2024-03-21T08:55:08.529049+0000","last_undegraded":"2024-03-21T08:55:08.529049+0000","last_fullsized":"2024-03-21T08:55:08.529049+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:54:47.093433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.528998+0000","last_change":"2024-03-21T08:55:07.536226+0000","last_active":"2024-03-21T08:55:08.528998+0000","last_peered":"2024-03-21T08:55:08.528998+0000","last_clean":"2024-03-21T08:55:08.528998+0000","last_became_active":"2024-03-21T08:55:07.535711+0000","last_became_peered":"2024-03-21T08:55:07.535711+0000","last_unstale":"2024-03-21T08:55:08.528998+0000","last_undegraded":"2024-03-21T08:55:08.528998+0000","last_fullsized":"2024-03-21T08:55:08.528998+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:37:29.252051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.540749+0000","last_change":"2024-03-21T08:55:07.536440+0000","last_active":"2024-03-21T08:55:07.540749+0000","last_peered":"2024-03-21T08:55:07.540749+0000","last_clean":"2024-03-21T08:55:07.540749+0000","last_became_active":"2024-03-21T08:55:07.536076+0000","last_became_peered":"2024-03-21T08:55:07.536076+0000","last_unstale":"2024-03-21T08:55:07.540749+0000","last_undegraded":"2024-03-21T08:55:07.540749+0000","last_fullsized":"2024-03-21T08:55:07.540749+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:01:29.420502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126426+0000","last_change":"2024-03-21T08:55:07.542634+0000","last_active":"2024-03-21T08:55:09.126426+0000","last_peered":"2024-03-21T08:55:09.126426+0000","last_clean":"2024-03-21T08:55:09.126426+0000","last_became_active":"2024-03-21T08:55:07.542317+0000","last_became_peered":"2024-03-21T08:55:07.542317+0000","last_unstale":"2024-03-21T08:55:09.126426+0000","last_undegraded":"2024-03-21T08:55:09.126426+0000","last_fullsized":"2024-03-21T08:55:09.126426+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:31.392606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286203+0000","last_change":"2024-03-21T08:55:07.537660+0000","last_active":"2024-03-21T08:55:09.286203+0000","last_peered":"2024-03-21T08:55:09.286203+0000","last_clean":"2024-03-21T08:55:09.286203+0000","last_became_active":"2024-03-21T08:55:07.537205+0000","last_became_peered":"2024-03-21T08:55:07.537205+0000","last_unstale":"2024-03-21T08:55:09.286203+0000","last_undegraded":"2024-03-21T08:55:09.286203+0000","last_fullsized":"2024-03-21T08:55:09.286203+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:39:59.336323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.529504+0000","last_change":"2024-03-21T08:55:07.530140+0000","last_active":"2024-03-21T08:55:08.529504+0000","last_peered":"2024-03-21T08:55:08.529504+0000","last_clean":"2024-03-21T08:55:08.529504+0000","last_became_active":"2024-03-21T08:55:07.529373+0000","last_became_peered":"2024-03-21T08:55:07.529373+0000","last_unstale":"2024-03-21T08:55:08.529504+0000","last_undegraded":"2024-03-21T08:55:08.529504+0000","last_fullsized":"2024-03-21T08:55:08.529504+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:37.939733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.946194+0000","last_change":"2024-03-21T08:55:07.529218+0000","last_active":"2024-03-21T08:55:08.946194+0000","last_peered":"2024-03-21T08:55:08.946194+0000","last_clean":"2024-03-21T08:55:08.946194+0000","last_became_active":"2024-03-21T08:55:07.528898+0000","last_became_peered":"2024-03-21T08:55:07.528898+0000","last_unstale":"2024-03-21T08:55:08.946194+0000","last_undegraded":"2024-03-21T08:55:08.946194+0000","last_fullsized":"2024-03-21T08:55:08.946194+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:49.392688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126165+0000","last_change":"2024-03-21T08:55:07.531920+0000","last_active":"2024-03-21T08:55:09.126165+0000","last_peered":"2024-03-21T08:55:09.126165+0000","last_clean":"2024-03-21T08:55:09.126165+0000","last_became_active":"2024-03-21T08:55:07.531266+0000","last_became_peered":"2024-03-21T08:55:07.531266+0000","last_unstale":"2024-03-21T08:55:09.126165+0000","last_undegraded":"2024-03-21T08:55:09.126165+0000","last_fullsized":"2024-03-21T08:55:09.126165+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:41:19.838525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:08.945794+0000","last_change":"2024-03-21T08:55:07.530658+0000","last_active":"2024-03-21T08:55:08.945794+0000","last_peered":"2024-03-21T08:55:08.945794+0000","last_clean":"2024-03-21T08:55:08.945794+0000","last_became_active":"2024-03-21T08:55:07.529933+0000","last_became_peered":"2024-03-21T08:55:07.529933+0000","last_unstale":"2024-03-21T08:55:08.945794+0000","last_undegraded":"2024-03-21T08:55:08.945794+0000","last_fullsized":"2024-03-21T08:55:08.945794+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:56.698603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-03-21T08:55:07.530269+0000","last_change":"2024-03-21T08:55:07.529151+0000","last_active":"2024-03-21T08:55:07.530269+0000","last_peered":"2024-03-21T08:55:07.530269+0000","last_clean":"2024-03-21T08:55:07.530269+0000","last_became_active":"2024-03-21T08:55:07.528511+0000","last_became_peered":"2024-03-21T08:55:07.528511+0000","last_unstale":"2024-03-21T08:55:07.530269+0000","last_undegraded":"2024-03-21T08:55:07.530269+0000","last_fullsized":"2024-03-21T08:55:07.530269+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:33:17.745591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.126468+0000","last_change":"2024-03-21T08:55:07.541519+0000","last_active":"2024-03-21T08:55:09.126468+0000","last_peered":"2024-03-21T08:55:09.126468+0000","last_clean":"2024-03-21T08:55:09.126468+0000","last_became_active":"2024-03-21T08:55:07.541171+0000","last_became_peered":"2024-03-21T08:55:07.541171+0000","last_unstale":"2024-03-21T08:55:09.126468+0000","last_undegraded":"2024-03-21T08:55:09.126468+0000","last_fullsized":"2024-03-21T08:55:09.126468+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:10:36.249778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-03-21T08:55:09.286028+0000","last_change":"2024-03-21T08:55:07.538455+0000","last_active":"2024-03-21T08:55:09.286028+0000","last_peered":"2024-03-21T08:55:09.286028+0000","last_clean":"2024-03-21T08:55:09.286028+0000","last_became_active":"2024-03-21T08:55:07.538082+0000","last_became_peered":"2024-03-21T08:55:07.538082+0000","last_unstale":"2024-03-21T08:55:09.286028+0000","last_undegraded":"2024-03-21T08:55:09.286028+0000","last_fullsized":"2024-03-21T08:55:09.286028+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:18:42.428381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280365,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":7439,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":9,"num_read_kb":7,"num_write":133,"num_write_kb":50,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":41,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":262144,"data_stored":24824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892350989,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":444207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612626,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1035189,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776152,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":929792,"data_stored":443597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037788,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":435049,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299427,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":999424,"data_stored":1036238,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462952,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32088,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713064,"statfs":{"total":95995035648,"available":95962177536,"internally_reserved":0,"allocated":933888,"data_stored":443127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574912,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":925696,"data_stored":436057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738437,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":917504,"data_stored":434535,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411388,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32124,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713028,"statfs":{"total":95995035648,"available":95962140672,"internally_reserved":0,"allocated":970752,"data_stored":446990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182454,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":950272,"data_stored":443423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920818,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":444072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724589,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":1035621,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":10184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2238,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:55:14.650 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-03-21T08:55:14.651 INFO:tasks.cephadm:Setting up client nodes... 2024-03-21T08:55:14.651 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-21T08:55:14.890 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:14 smithi105 ceph-mon[35993]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:15.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:14 smithi105 ceph-mon[35993]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-21T08:55:15.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:14 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1406855185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:15.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:14 smithi100 ceph-mon[36306]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:15.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:14 smithi100 ceph-mon[36306]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-21T08:55:15.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:14 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1406855185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:15.290 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:14 smithi087 ceph-mon[32385]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:55:15.290 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:14 smithi087 ceph-mon[32385]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-03-21T08:55:15.290 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:14 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1406855185' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:16.062 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:15 smithi087 ceph-mon[32385]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:55:16.062 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:15 smithi087 ceph-mon[32385]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:55:16.062 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:15 smithi087 ceph-mon[32385]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 KiB/s wr, 31 op/s 2024-03-21T08:55:16.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:15 smithi105 ceph-mon[35993]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:55:16.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:15 smithi105 ceph-mon[35993]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:55:16.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:15 smithi105 ceph-mon[35993]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 KiB/s wr, 31 op/s 2024-03-21T08:55:16.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:15 smithi100 ceph-mon[36306]: from='client.14796 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:55:16.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:15 smithi100 ceph-mon[36306]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:55:16.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:15 smithi100 ceph-mon[36306]: pgmap v294: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 13 KiB/s wr, 31 op/s 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 -- 172.21.15.87:0/2025226347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101d00 msgr2=0x7f8bdc1095a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/2025226347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101d00 0x7f8bdc1095a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8bd400b0a0 tx=0x7f8bd400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 -- 172.21.15.87:0/2025226347 shutdown_connections 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/2025226347 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc1026e0 0x7f8bdc102b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/2025226347 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101d00 0x7f8bdc1095a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/2025226347 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc101340 0x7f8bdc1017c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.644 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 -- 172.21.15.87:0/2025226347 >> 172.21.15.87:0/2025226347 conn(0x7f8bdc0fbe30 msgr2=0x7f8bdc0fe250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 -- 172.21.15.87:0/2025226347 shutdown_connections 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.645+0000 7f8be1dbd700 1 -- 172.21.15.87:0/2025226347 wait complete. 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 Processor -- start 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 -- start start 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 0x7f8bdc105090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc101d00 0x7f8bdc103730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc1026e0 0x7f8bdc103c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc10cd00 con 0x7f8bdc101d00 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc10cb80 con 0x7f8bdc1026e0 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8be1dbd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bdc10ce80 con 0x7f8bdc101340 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8bdb7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 0x7f8bdc105090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8bdb7fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 0x7f8bdc105090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:50876/0 (socket says 172.21.15.87:50876) 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8bdb7fe700 1 -- 172.21.15.87:0/1904285298 learned_addr learned my addr 172.21.15.87:0/1904285298 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:16.645 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8bdaffd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc101d00 0x7f8bdc103730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.646+0000 7f8bdbfff700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc1026e0 0x7f8bdc103c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bdb7fe700 1 -- 172.21.15.87:0/1904285298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc1026e0 msgr2=0x7f8bdc103c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bdb7fe700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc1026e0 0x7f8bdc103c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bdb7fe700 1 -- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc101d00 msgr2=0x7f8bdc103730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bdb7fe700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc101d00 0x7f8bdc103730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bdb7fe700 1 -- 172.21.15.87:0/1904285298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bdc104540 con 0x7f8bdc101340 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bdb7fe700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 0x7f8bdc105090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e950 tx=0x7f8bcc00ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc004230 con 0x7f8bdc101340 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8bcc0043d0 con 0x7f8bdc101340 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bdc1a2400 con 0x7f8bdc101340 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc003dd0 con 0x7f8bdc101340 2024-03-21T08:55:16.646 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.647+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bdc1a28c0 con 0x7f8bdc101340 2024-03-21T08:55:16.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.648+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bdc10d460 con 0x7f8bdc101340 2024-03-21T08:55:16.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.649+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8bcc00c7e0 con 0x7f8bdc101340 2024-03-21T08:55:16.647 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.650+0000 7f8bd8ff9700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8bc40a30f0 0x7f8bc40a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:16.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.650+0000 7f8bdaffd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8bc40a30f0 0x7f8bc40a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:16.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.650+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8bcc0cf700 con 0x7f8bdc101340 2024-03-21T08:55:16.648 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.650+0000 7f8bdaffd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8bc40a30f0 0x7f8bc40a55b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009c60 tx=0x7f8bd0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:16.650 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.654+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8bcc09c500 con 0x7f8bdc101340 2024-03-21T08:55:16.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.928+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f8bdc1a2d40 con 0x7f8bdc101340 2024-03-21T08:55:16.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.932+0000 7f8bd8ff9700 1 -- 172.21.15.87:0/1904285298 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7f8bdc1a2d40 con 0x7f8bdc101340 2024-03-21T08:55:16.928 INFO:teuthology.orchestra.run.smithi087.stdout:[client.0] 2024-03-21T08:55:16.929 INFO:teuthology.orchestra.run.smithi087.stdout: key = AQD09ftlg2dwNxAACsi48Ym0cdIXsAiJpvLquQ== 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8bc40a30f0 msgr2=0x7f8bc40a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8bc40a30f0 0x7f8bc40a55b0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009c60 tx=0x7f8bd0009260 comp rx=0 tx=0).stop 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 msgr2=0x7f8bdc105090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 0x7f8bdc105090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00e950 tx=0x7f8bcc00ec60 comp rx=0 tx=0).stop 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 shutdown_connections 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8bc40a30f0 0x7f8bc40a55b0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8bdc1026e0 0x7f8bdc103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8bdc101340 0x7f8bdc105090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 --2- 172.21.15.87:0/1904285298 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8bdc101d00 0x7f8bdc103730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 >> 172.21.15.87:0/1904285298 conn(0x7f8bdc0fbe30 msgr2=0x7f8bdc0fd760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:16.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 shutdown_connections 2024-03-21T08:55:16.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:16.934+0000 7f8be1dbd700 1 -- 172.21.15.87:0/1904285298 wait complete. 2024-03-21T08:55:17.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:16 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1904285298' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:55:17.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:16 smithi105 ceph-mon[35993]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:55:17.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:16 smithi105 ceph-mon[35993]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T08:55:17.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:16 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1904285298' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:55:17.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:16 smithi100 ceph-mon[36306]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:55:17.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:16 smithi100 ceph-mon[36306]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T08:55:17.273 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:16 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1904285298' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:55:17.273 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:16 smithi087 ceph-mon[32385]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-21T08:55:17.273 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:16 smithi087 ceph-mon[32385]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-21T08:55:17.590 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:55:17.590 DEBUG:teuthology.orchestra.run.smithi087:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-21T08:55:17.590 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-21T08:55:17.673 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-21T08:55:17.673 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-21T08:55:17.673 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph mgr dump --format=json 2024-03-21T08:55:17.952 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:18.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:17 smithi105 ceph-mon[35993]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-03-21T08:55:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:17 smithi100 ceph-mon[36306]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-03-21T08:55:18.315 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:17 smithi087 ceph-mon[32385]: pgmap v295: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 11 KiB/s wr, 27 op/s 2024-03-21T08:55:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2015261792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f4102d80 msgr2=0x7fb4f4106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2015261792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f4102d80 0x7fb4f4106dd0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4009990 tx=0x7fb4e4009ca0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2015261792 shutdown_connections 2024-03-21T08:55:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2015261792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41074b0 0x7fb4f410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2015261792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f41023d0 0x7fb4f41027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2015261792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f4102d80 0x7fb4f4106dd0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2015261792 >> 172.21.15.87:0/2015261792 conn(0x7fb4f40fdc40 msgr2=0x7fb4f4100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2015261792 shutdown_connections 2024-03-21T08:55:19.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.719+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2015261792 wait complete. 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.720+0000 7fb4fa9d0700 1 Processor -- start 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.720+0000 7fb4fa9d0700 1 -- start start 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.720+0000 7fb4fa9d0700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 0x7fb4f419e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.720+0000 7fb4fa9d0700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f4102d80 0x7fb4f419eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.720+0000 7fb4fa9d0700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f41074b0 0x7fb4f4198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4fa9d0700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f4112470 con 0x7fb4f41023d0 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4fa9d0700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41122f0 con 0x7fb4f41074b0 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4fa9d0700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f41125f0 con 0x7fb4f4102d80 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 0x7fb4f419e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:19.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 0x7fb4f419e610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52422/0 (socket says 172.21.15.87:52422) 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 -- 172.21.15.87:0/2925437049 learned_addr learned my addr 172.21.15.87:0/2925437049 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f37fe700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f4102d80 0x7fb4f419eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f8f6d700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f41074b0 0x7fb4f4198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 -- 172.21.15.87:0/2925437049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f4102d80 msgr2=0x7fb4f419eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f4102d80 0x7fb4f419eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 -- 172.21.15.87:0/2925437049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f41074b0 msgr2=0x7fb4f4198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f41074b0 0x7fb4f4198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 -- 172.21.15.87:0/2925437049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f4198f70 con 0x7fb4f41023d0 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f37fe700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f4102d80 0x7fb4f419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f3fff700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 0x7fb4f419e610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00eb30 tx=0x7fb4dc00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc00cbd0 con 0x7fb4f41023d0 2024-03-21T08:55:19.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f4199260 con 0x7fb4f41023d0 2024-03-21T08:55:19.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb4dc004500 con 0x7fb4f41023d0 2024-03-21T08:55:19.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4dc010430 con 0x7fb4f41023d0 2024-03-21T08:55:19.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.721+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f41a5470 con 0x7fb4f41023d0 2024-03-21T08:55:19.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.722+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f41038b0 con 0x7fb4f41023d0 2024-03-21T08:55:19.721 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.723+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb4dc003f00 con 0x7fb4f41023d0 2024-03-21T08:55:19.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.724+0000 7fb4f17fa700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e00a3040 0x7fb4e00a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:19.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.724+0000 7fb4f37fe700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e00a3040 0x7fb4e00a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:19.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.724+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb4dc0cf690 con 0x7fb4f41023d0 2024-03-21T08:55:19.722 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.724+0000 7fb4f37fe700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e00a3040 0x7fb4e00a5500 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006010 tx=0x7fb4e401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:19.723 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.728+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb4dc09c5a0 con 0x7fb4f41023d0 2024-03-21T08:55:19.977 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.981+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fb4f402da30 con 0x7fb4f41023d0 2024-03-21T08:55:19.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.985+0000 7fb4f17fa700 1 -- 172.21.15.87:0/2925437049 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+263550 (secure 0 0 0) 0x7fb4dc09bea0 con 0x7fb4f41023d0 2024-03-21T08:55:19.986 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.989+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e00a3040 msgr2=0x7fb4e00a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.989+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e00a3040 0x7fb4e00a5500 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4e4006010 tx=0x7fb4e401a040 comp rx=0 tx=0).stop 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.989+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 msgr2=0x7fb4f419e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.989+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 0x7fb4f419e610 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc00eb30 tx=0x7fb4dc00ee40 comp rx=0 tx=0).stop 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 shutdown_connections 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb4e00a3040 0x7fb4e00a5500 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb4f41074b0 0x7fb4f4198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb4f41023d0 0x7fb4f419e610 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 --2- 172.21.15.87:0/2925437049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb4f4102d80 0x7fb4f419eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:19.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 >> 172.21.15.87:0/2925437049 conn(0x7fb4f40fdc40 msgr2=0x7fb4f40fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:19.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 shutdown_connections 2024-03-21T08:55:19.989 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:19.990+0000 7fb4fa9d0700 1 -- 172.21.15.87:0/2925437049 wait complete. 2024-03-21T08:55:20.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:19 smithi105 ceph-mon[35993]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-03-21T08:55:20.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:19 smithi100 ceph-mon[36306]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-03-21T08:55:20.378 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:19 smithi087 ceph-mon[32385]: pgmap v296: 97 pgs: 97 active+clean; 621 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 10 KiB/s wr, 27 op/s 2024-03-21T08:55:20.749 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6800","nonce":1005139006},{"type":"v1","addr":"172.21.15.87:6801","nonce":1005139006}]},"active_addr":"172.21.15.87:6801/1005139006","active_change":"2024-03-21T08:47:58.771601+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.87:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":671247144}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":1056206501}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":924709719}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3098728466}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.87:0","nonce":3968949730}]}]} 2024-03-21T08:55:20.754 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-21T08:55:20.754 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-21T08:55:20.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T08:55:20.995 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:21.222 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:20 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2925437049' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T08:55:21.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:20 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2925437049' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T08:55:21.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:20 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2925437049' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-21T08:55:22.138 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:21 smithi087 ceph-mon[32385]: pgmap v297: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-03-21T08:55:22.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:21 smithi105 ceph-mon[35993]: pgmap v297: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-03-21T08:55:22.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:21 smithi100 ceph-mon[36306]: pgmap v297: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 9.2 KiB/s wr, 25 op/s 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 -- 172.21.15.87:0/3437148497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc101a30 msgr2=0x7f24fc10df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 --2- 172.21.15.87:0/3437148497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc101a30 0x7f24fc10df10 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b0a0 tx=0x7f24f400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 -- 172.21.15.87:0/3437148497 shutdown_connections 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 --2- 172.21.15.87:0/3437148497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc101a30 0x7f24fc10df10 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 --2- 172.21.15.87:0/3437148497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc1070d0 0x7f24fc1074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 --2- 172.21.15.87:0/3437148497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc101090 0x7f24fc1014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.709+0000 7f2502427700 1 -- 172.21.15.87:0/3437148497 >> 172.21.15.87:0/3437148497 conn(0x7f24fc077fc0 msgr2=0x7f24fc0783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.710+0000 7f2502427700 1 -- 172.21.15.87:0/3437148497 shutdown_connections 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.710+0000 7f2502427700 1 -- 172.21.15.87:0/3437148497 wait complete. 2024-03-21T08:55:22.708 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.710+0000 7f2502427700 1 Processor -- start 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.710+0000 7f2502427700 1 -- start start 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc101090 0x7f24fc199700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc101a30 0x7f24fc199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 0x7f24fc1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc111320 con 0x7f24fc1070d0 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1111a0 con 0x7f24fc101a30 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24fc1114a0 con 0x7f24fc101090 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 0x7f24fc1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 0x7f24fc1a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52450/0 (socket says 172.21.15.87:52450) 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 -- 172.21.15.87:0/517636402 learned_addr learned my addr 172.21.15.87:0/517636402 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f24fbfff700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc101090 0x7f24fc199700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f24fb7fe700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc101a30 0x7f24fc199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 -- 172.21.15.87:0/517636402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc101090 msgr2=0x7f24fc199700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:22.709 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc101090 0x7f24fc199700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 -- 172.21.15.87:0/517636402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc101a30 msgr2=0x7f24fc199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc101a30 0x7f24fc199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 -- 172.21.15.87:0/517636402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc1a4cc0 con 0x7f24fc1070d0 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f24fbfff700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc101090 0x7f24fc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f25009c4700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 0x7f24fc1a4780 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b780 tx=0x7f24f4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f4021070 con 0x7f24fc1070d0 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24fc1a4e90 con 0x7f24fc1070d0 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.711+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f24f4007b80 con 0x7f24fc1070d0 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.712+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24f40052a0 con 0x7f24fc1070d0 2024-03-21T08:55:22.710 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.712+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24fc1a5390 con 0x7f24fc1070d0 2024-03-21T08:55:22.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.713+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24fc068e50 con 0x7f24fc1070d0 2024-03-21T08:55:22.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.714+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f24f4024070 con 0x7f24fc1070d0 2024-03-21T08:55:22.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.714+0000 7f24f97fa700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f24e40a3040 0x7f24e40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:22.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.714+0000 7f24fbfff700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f24e40a3040 0x7f24e40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:22.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.714+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24f400f020 con 0x7f24fc1070d0 2024-03-21T08:55:22.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.715+0000 7f24fbfff700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f24e40a3040 0x7f24e40a5500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009fd0 tx=0x7f24ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:22.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.719+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f24f402b020 con 0x7f24fc1070d0 2024-03-21T08:55:22.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.924+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24fc02da30 con 0x7f24fc1070d0 2024-03-21T08:55:22.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.926+0000 7f24f97fa700 1 -- 172.21.15.87:0/517636402 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22229 (secure 0 0 0) 0x7f24f4007d30 con 0x7f24fc1070d0 2024-03-21T08:55:22.923 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:22.923 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":86,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:55:08.517781+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:55:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f24e40a3040 msgr2=0x7f24e40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:22.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f24e40a3040 0x7f24e40a5500 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009fd0 tx=0x7f24ec009260 comp rx=0 tx=0).stop 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 msgr2=0x7f24fc1a4780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 0x7f24fc1a4780 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f24f400b780 tx=0x7f24f4004d00 comp rx=0 tx=0).stop 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 shutdown_connections 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f24e40a3040 0x7f24e40a5500 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f24fc1070d0 0x7f24fc1a4780 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f24fc101090 0x7f24fc199700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 --2- 172.21.15.87:0/517636402 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f24fc101a30 0x7f24fc199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 >> 172.21.15.87:0/517636402 conn(0x7f24fc077fc0 msgr2=0x7f24fc0ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 shutdown_connections 2024-03-21T08:55:22.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:22.928+0000 7f2502427700 1 -- 172.21.15.87:0/517636402 wait complete. 2024-03-21T08:55:23.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:22 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/517636402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:23.304 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:22 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/517636402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:23.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:22 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/517636402' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:23.648 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-21T08:55:23.649 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd dump --format=json 2024-03-21T08:55:23.891 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:24.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:23 smithi100 ceph-mon[36306]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.8 KiB/s wr, 26 op/s 2024-03-21T08:55:24.284 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:23 smithi087 ceph-mon[32385]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.8 KiB/s wr, 26 op/s 2024-03-21T08:55:24.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:23 smithi105 ceph-mon[35993]: pgmap v298: 97 pgs: 97 active+clean; 626 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.9 KiB/s rd, 8.8 KiB/s wr, 26 op/s 2024-03-21T08:55:25.685 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 -- 172.21.15.87:0/2447420206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f01070d0 msgr2=0x7f50f01074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 --2- 172.21.15.87:0/2447420206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f01070d0 0x7f50f01074b0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f50e0009990 tx=0x7f50e0009ca0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 -- 172.21.15.87:0/2447420206 shutdown_connections 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 --2- 172.21.15.87:0/2447420206 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f0101a30 0x7f50f010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 --2- 172.21.15.87:0/2447420206 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f01070d0 0x7f50f01074b0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 --2- 172.21.15.87:0/2447420206 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50f0101090 0x7f50f01014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 -- 172.21.15.87:0/2447420206 >> 172.21.15.87:0/2447420206 conn(0x7f50f0077fc0 msgr2=0x7f50f00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 -- 172.21.15.87:0/2447420206 shutdown_connections 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.681+0000 7f50f4f30700 1 -- 172.21.15.87:0/2447420206 wait complete. 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 Processor -- start 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 -- start start 2024-03-21T08:55:25.686 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 0x7f50f0197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f0101a30 0x7f50f0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50f01070d0 0x7f50f0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f0111330 con 0x7f50f0101090 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01111b0 con 0x7f50f0101a30 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50f4f30700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50f01114b0 con 0x7f50f01070d0 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 0x7f50f0197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 0x7f50f0197540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:52458/0 (socket says 172.21.15.87:52458) 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 -- 172.21.15.87:0/73977863 learned_addr learned my addr 172.21.15.87:0/73977863 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50edd9b700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f0101a30 0x7f50f0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50eed9d700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50f01070d0 0x7f50f0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 -- 172.21.15.87:0/73977863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50f01070d0 msgr2=0x7f50f0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50f01070d0 0x7f50f0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 -- 172.21.15.87:0/73977863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f0101a30 msgr2=0x7f50f0197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:25.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f0101a30 0x7f50f0197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 -- 172.21.15.87:0/73977863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f01a6db0 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50ee59c700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 0x7f50f0197540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50e0000c00 tx=0x7f50e000bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.682+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e0027070 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.683+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50f01a7040 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.683+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f50e0003d70 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.683+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50e000f460 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.683+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f01a74f0 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.684+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50f0068e50 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.685+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f50e002a070 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.685+0000 7f50df7fe700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f50d80a3040 0x7f50d80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.685+0000 7f50edd9b700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f50d80a3040 0x7f50d80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.686+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f50e00d0200 con 0x7f50f0101090 2024-03-21T08:55:25.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.686+0000 7f50edd9b700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f50d80a3040 0x7f50d80a5500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50f01996a0 tx=0x7f50e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:25.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.690+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f50e0013070 con 0x7f50f0101090 2024-03-21T08:55:25.882 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.886+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50f002da30 con 0x7f50f0101090 2024-03-21T08:55:25.884 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.888+0000 7f50df7fe700 1 -- 172.21.15.87:0/73977863 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+22229 (secure 0 0 0) 0x7f50e009d110 con 0x7f50f0101090 2024-03-21T08:55:25.884 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:55:25.884 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":86,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:55:08.517781+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:55:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f50d80a3040 msgr2=0x7f50d80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f50d80a3040 0x7f50d80a5500 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f50f01996a0 tx=0x7f50e4009260 comp rx=0 tx=0).stop 2024-03-21T08:55:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 msgr2=0x7f50f0197540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 0x7f50f0197540 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f50e0000c00 tx=0x7f50e000bab0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 shutdown_connections 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f50d80a3040 0x7f50d80a5500 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f50f01070d0 0x7f50f0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f50f0101090 0x7f50f0197540 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 --2- 172.21.15.87:0/73977863 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f50f0101a30 0x7f50f0197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 >> 172.21.15.87:0/73977863 conn(0x7f50f0077fc0 msgr2=0x7f50f00ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 shutdown_connections 2024-03-21T08:55:25.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:25.890+0000 7f50f4f30700 1 -- 172.21.15.87:0/73977863 wait complete. 2024-03-21T08:55:26.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:25 smithi087 ceph-mon[32385]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-03-21T08:55:26.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:25 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/73977863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:26.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:25 smithi100 ceph-mon[36306]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-03-21T08:55:26.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:25 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/73977863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:26.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:25 smithi105 ceph-mon[35993]: pgmap v299: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 6.4 KiB/s wr, 20 op/s 2024-03-21T08:55:26.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:25 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/73977863' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:55:26.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-21T08:55:26.685 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-21T08:55:26.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-21T08:55:26.686 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-21T08:55:26.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-21T08:55:26.687 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-21T08:55:26.688 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-21T08:55:26.688 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-21T08:55:26.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2024-03-21T08:55:26.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2024-03-21T08:55:26.689 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2024-03-21T08:55:26.690 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2024-03-21T08:55:27.268 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.045 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.051 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.051 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.052 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.054 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.055 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:27 smithi100 ceph-mon[36306]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-03-21T08:55:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:27 smithi087 ceph-mon[32385]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-03-21T08:55:28.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:27 smithi105 ceph-mon[35993]: pgmap v300: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.9 KiB/s wr, 11 op/s 2024-03-21T08:55:30.269 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:29 smithi100 ceph-mon[36306]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-03-21T08:55:30.326 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:29 smithi087 ceph-mon[32385]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-03-21T08:55:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:29 smithi105 ceph-mon[35993]: pgmap v301: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.9 KiB/s wr, 13 op/s 2024-03-21T08:55:30.653 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.652+0000 7f2c35780700 1 -- 172.21.15.87:0/2055740762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101320 msgr2=0x7f2c30101700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.652+0000 7f2c35780700 1 --2- 172.21.15.87:0/2055740762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101320 0x7f2c30101700 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2c200099c0 tx=0x7f2c20009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.652+0000 7f2c35780700 1 -- 172.21.15.87:0/2055740762 shutdown_connections 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.652+0000 7f2c35780700 1 --2- 172.21.15.87:0/2055740762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30106370 0x7f2c3010df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.652+0000 7f2c35780700 1 --2- 172.21.15.87:0/2055740762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30101320 0x7f2c30101700 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.652+0000 7f2c35780700 1 --2- 172.21.15.87:0/2055740762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30101cd0 0x7f2c30105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 -- 172.21.15.87:0/2055740762 >> 172.21.15.87:0/2055740762 conn(0x7f2c300770e0 msgr2=0x7f2c300774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 -- 172.21.15.87:0/2055740762 shutdown_connections 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 -- 172.21.15.87:0/2055740762 wait complete. 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 Processor -- start 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 -- start start 2024-03-21T08:55:30.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30101320 0x7f2c30102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.653+0000 7f2c35780700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30101cd0 0x7f2c30103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c35780700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30106370 0x7f2c301037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c35780700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30111380 con 0x7f2c30106370 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c35780700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30111200 con 0x7f2c30101cd0 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c35780700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c30111500 con 0x7f2c30101320 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2effd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30101320 0x7f2c30102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30106370 0x7f2c301037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2effd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30101320 0x7f2c30102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:44526/0 (socket says 172.21.15.87:44526) 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2e7fc700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30101cd0 0x7f2c30103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2effd700 1 -- 172.21.15.87:0/316939248 learned_addr learned my addr 172.21.15.87:0/316939248 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 -- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30101320 msgr2=0x7f2c30102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30101320 0x7f2c30102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 -- 172.21.15.87:0/316939248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30101cd0 msgr2=0x7f2c30103270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30101cd0 0x7f2c30103270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 -- 172.21.15.87:0/316939248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c301a6f40 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2e7fc700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c30101cd0 0x7f2c30103270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2effd700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c30101320 0x7f2c30102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c2f7fe700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30106370 0x7f2c301037b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800efc0 tx=0x7f2c2800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280092e0 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c301a71d0 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c28004bb0 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28004280 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.654+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c301a7710 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.655+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c30062860 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.656+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2c2801b030 con 0x7f2c30106370 2024-03-21T08:55:30.656 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c17fff700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c180a3040 0x7f2c180a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c2effd700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c180a3040 0x7f2c180a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c280cf820 con 0x7f2c30106370 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c17fff700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] conn(0x7f2c180ac820 0x7f2c180aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 --> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c28010ec0 con 0x7f2c180ac820 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c280cfd00 con 0x7f2c30106370 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c2effd700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c180a3040 0x7f2c180a5500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009990 tx=0x7f2c20005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.657+0000 7f2c2e7fc700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] conn(0x7f2c180ac820 0x7f2c180aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:30.657 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.658+0000 7f2c2e7fc700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] conn(0x7f2c180ac820 0x7f2c180aec40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:30.658 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.658+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== osd.5 v2:172.21.15.105:6808/1039939293 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2c28010ec0 con 0x7f2c180ac820 2024-03-21T08:55:30.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.676+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 --> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c30062c90 con 0x7f2c180ac820 2024-03-21T08:55:30.671 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.676+0000 7f2c17fff700 1 -- 172.21.15.87:0/316939248 <== osd.5 v2:172.21.15.105:6808/1039939293 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f2c30062c90 con 0x7f2c180ac820 2024-03-21T08:55:30.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.676+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] conn(0x7f2c180ac820 msgr2=0x7f2c180aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:30.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.676+0000 7f2c35780700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] conn(0x7f2c180ac820 0x7f2c180aec40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c180a3040 msgr2=0x7f2c180a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:30.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c180a3040 0x7f2c180a5500 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009990 tx=0x7f2c20005960 comp rx=0 tx=0).stop 2024-03-21T08:55:30.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30106370 msgr2=0x7f2c301037b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:30.672 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 --2- 172.21.15.87:0/316939248 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c30106370 0x7f2c301037b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800efc0 tx=0x7f2c2800c3a0 comp rx=0 tx=0).stop 2024-03-21T08:55:30.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c2f7fe700 1 -- 172.21.15.87:0/316939248 reap_dead start 2024-03-21T08:55:30.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 shutdown_connections 2024-03-21T08:55:30.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 >> 172.21.15.87:0/316939248 conn(0x7f2c300770e0 msgr2=0x7f2c3006aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:30.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 shutdown_connections 2024-03-21T08:55:30.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:30.677+0000 7f2c35780700 1 -- 172.21.15.87:0/316939248 wait complete. 2024-03-21T08:55:31.547 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:32.041 INFO:teuthology.orchestra.run.smithi087.stdout:167503724593 2024-03-21T08:55:32.042 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-21T08:55:32.182 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:31 smithi087 ceph-mon[32385]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-21T08:55:32.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:31 smithi100 ceph-mon[36306]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-21T08:55:32.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:31 smithi105 ceph-mon[35993]: pgmap v302: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 11 op/s 2024-03-21T08:55:32.757 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:34.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:34 smithi087 ceph-mon[32385]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T08:55:34.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:34 smithi100 ceph-mon[36306]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T08:55:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:34 smithi105 ceph-mon[35993]: pgmap v303: 97 pgs: 97 active+clean; 638 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-03-21T08:55:35.730 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:36.147 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:36 smithi087 ceph-mon[32385]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T08:55:36.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:36 smithi100 ceph-mon[36306]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 -- 172.21.15.87:0/361305960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810100d20 msgr2=0x7fc810101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 --2- 172.21.15.87:0/361305960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810100d20 0x7fc810101100 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc80c0099c0 tx=0x7fc80c009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 -- 172.21.15.87:0/361305960 shutdown_connections 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 --2- 172.21.15.87:0/361305960 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc810107540 0x7fc81010f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 --2- 172.21.15.87:0/361305960 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810100d20 0x7fc810101100 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 --2- 172.21.15.87:0/361305960 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8101016d0 0x7fc810106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.347+0000 7fc817eea700 1 -- 172.21.15.87:0/361305960 >> 172.21.15.87:0/361305960 conn(0x7fc8100763c0 msgr2=0x7fc8100767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 -- 172.21.15.87:0/361305960 shutdown_connections 2024-03-21T08:55:36.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 -- 172.21.15.87:0/361305960 wait complete. 2024-03-21T08:55:36.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 Processor -- start 2024-03-21T08:55:36.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 -- start start 2024-03-21T08:55:36.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810100d20 0x7fc810101ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8101016d0 0x7fc8101021e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810107540 0x7fc810102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8101124f0 con 0x7fc810107540 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810112370 con 0x7fc8101016d0 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.348+0000 7fc817eea700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc810112670 con 0x7fc810100d20 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810107540 0x7fc810102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc815c86700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810100d20 0x7fc810101ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810107540 0x7fc810102720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49358/0 (socket says 172.21.15.87:49358) 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 -- 172.21.15.87:0/664242687 learned_addr learned my addr 172.21.15.87:0/664242687 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc815485700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8101016d0 0x7fc8101021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 -- 172.21.15.87:0/664242687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810100d20 msgr2=0x7fc810101ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:36.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810100d20 0x7fc810101ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 -- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8101016d0 msgr2=0x7fc8101021e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8101016d0 0x7fc8101021e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 -- 172.21.15.87:0/664242687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8101a6eb0 con 0x7fc810107540 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc815c86700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc810100d20 0x7fc810101ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc816487700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810107540 0x7fc810102720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc80800ef30 tx=0x7fc80800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080092e0 con 0x7fc810107540 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8101a7140 con 0x7fc810107540 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc808004bb0 con 0x7fc810107540 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808004280 con 0x7fc810107540 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.349+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8101a76d0 con 0x7fc810107540 2024-03-21T08:55:36.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.350+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8100628a0 con 0x7fc810107540 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.351+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fc80801b030 con 0x7fc810107540 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc806ffd700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc7fc0a3040 0x7fc7fc0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc815c86700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc7fc0a3040 0x7fc7fc0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc8080cf7c0 con 0x7fc810107540 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc806ffd700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] conn(0x7fc7fc0ac820 0x7fc7fc0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 --> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7fc0af2f0 con 0x7fc7fc0ac820 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8080cfca0 con 0x7fc810107540 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc815485700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] conn(0x7fc7fc0ac820 0x7fc7fc0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.352+0000 7fc815c86700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc7fc0a3040 0x7fc7fc0a5500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000c00 tx=0x7fc80c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.353+0000 7fc815485700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] conn(0x7fc7fc0ac820 0x7fc7fc0aec40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:36.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.354+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== osd.10 v2:172.21.15.100:6824/1731228246 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc7fc0af2f0 con 0x7fc7fc0ac820 2024-03-21T08:55:36.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.370+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 --> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc810062ca0 con 0x7fc7fc0ac820 2024-03-21T08:55:36.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.370+0000 7fc806ffd700 1 -- 172.21.15.87:0/664242687 <== osd.10 v2:172.21.15.100:6824/1731228246 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc810062ca0 con 0x7fc7fc0ac820 2024-03-21T08:55:36.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.370+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] conn(0x7fc7fc0ac820 msgr2=0x7fc7fc0aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:36.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.370+0000 7fc817eea700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] conn(0x7fc7fc0ac820 0x7fc7fc0aec40 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc7fc0a3040 msgr2=0x7fc7fc0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:36.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc7fc0a3040 0x7fc7fc0a5500 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc80c000c00 tx=0x7fc80c005fb0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810107540 msgr2=0x7fc810102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 --2- 172.21.15.87:0/664242687 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc810107540 0x7fc810102720 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc80800ef30 tx=0x7fc80800c3a0 comp rx=0 tx=0).stop 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc816487700 1 -- 172.21.15.87:0/664242687 reap_dead start 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 shutdown_connections 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 >> 172.21.15.87:0/664242687 conn(0x7fc8100763c0 msgr2=0x7fc8100feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 shutdown_connections 2024-03-21T08:55:36.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.371+0000 7fc817eea700 1 -- 172.21.15.87:0/664242687 wait complete. 2024-03-21T08:55:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:36 smithi105 ceph-mon[35993]: pgmap v304: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T08:55:37.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 -- 172.21.15.87:0/1710063487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101090 msgr2=0x7fa9c01014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:37.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/1710063487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101090 0x7fa9c01014f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa9b4009990 tx=0x7fa9b4009ca0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 -- 172.21.15.87:0/1710063487 shutdown_connections 2024-03-21T08:55:37.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/1710063487 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c01070d0 0x7fa9c01074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/1710063487 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9c0101a30 0x7fa9c010df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/1710063487 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101090 0x7fa9c01014f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 -- 172.21.15.87:0/1710063487 >> 172.21.15.87:0/1710063487 conn(0x7fa9c0077fc0 msgr2=0x7fa9c00783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:37.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 -- 172.21.15.87:0/1710063487 shutdown_connections 2024-03-21T08:55:37.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 -- 172.21.15.87:0/1710063487 wait complete. 2024-03-21T08:55:37.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 Processor -- start 2024-03-21T08:55:37.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.573+0000 7fa9c66d1700 1 -- start start 2024-03-21T08:55:37.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9c0101090 0x7fa9c019e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:37.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101a30 0x7fa9c019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:37.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c01070d0 0x7fa9c0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:37.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c0111330 con 0x7fa9c0101a30 2024-03-21T08:55:37.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c01111b0 con 0x7fa9c01070d0 2024-03-21T08:55:37.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9c01114b0 con 0x7fa9c0101090 2024-03-21T08:55:37.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101a30 0x7fa9c019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:37.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101a30 0x7fa9c019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49370/0 (socket says 172.21.15.87:49370) 2024-03-21T08:55:37.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 -- 172.21.15.87:0/3248163068 learned_addr learned my addr 172.21.15.87:0/3248163068 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bffff700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9c0101090 0x7fa9c019e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c4c6e700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c01070d0 0x7fa9c0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 -- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9c0101090 msgr2=0x7fa9c019e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9c0101090 0x7fa9c019e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 -- 172.21.15.87:0/3248163068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c01070d0 msgr2=0x7fa9c0198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9c01070d0 0x7fa9c0198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.093 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 -- 172.21.15.87:0/3248163068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0199020 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bf7fe700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101a30 0x7fa9c019ebb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bae0 tx=0x7fa9b400bb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b4026070 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9c01992b0 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9b4003d70 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9b400fe50 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.574+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9c01a5420 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.575+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa9c0105190 con 0x7fa9c0101a30 2024-03-21T08:55:37.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.576+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa9b4029070 con 0x7fa9c0101a30 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bd7fa700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9a80a31e0 0x7fa9a80a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bffff700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9a80a31e0 0x7fa9a80a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fa9b40d0870 con 0x7fa9c0101a30 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bd7fa700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] conn(0x7fa9a80ac9c0 0x7fa9a80aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 --> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9b402ce50 con 0x7fa9a80ac9c0 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa9b40d0d50 con 0x7fa9c0101a30 2024-03-21T08:55:37.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.577+0000 7fa9bffff700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9a80a31e0 0x7fa9a80a56a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009680 tx=0x7fa9b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:37.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.578+0000 7fa9c4c6e700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] conn(0x7fa9a80ac9c0 0x7fa9a80aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:37.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.578+0000 7fa9c4c6e700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] conn(0x7fa9a80ac9c0 0x7fa9a80aede0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:37.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.579+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== osd.2 v2:172.21.15.105:6800/3040563214 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fa9b402ce50 con 0x7fa9a80ac9c0 2024-03-21T08:55:37.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.596+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 --> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa9c0102780 con 0x7fa9a80ac9c0 2024-03-21T08:55:37.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.596+0000 7fa9bd7fa700 1 -- 172.21.15.87:0/3248163068 <== osd.2 v2:172.21.15.105:6800/3040563214 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa9c0102780 con 0x7fa9a80ac9c0 2024-03-21T08:55:37.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] conn(0x7fa9a80ac9c0 msgr2=0x7fa9a80aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:37.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] conn(0x7fa9a80ac9c0 0x7fa9a80aede0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:37.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9a80a31e0 msgr2=0x7fa9a80a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:37.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9a80a31e0 0x7fa9a80a56a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0009680 tx=0x7fa9b0009260 comp rx=0 tx=0).stop 2024-03-21T08:55:37.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101a30 msgr2=0x7fa9c019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:37.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 --2- 172.21.15.87:0/3248163068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9c0101a30 0x7fa9c019ebb0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa9b400bae0 tx=0x7fa9b400bb10 comp rx=0 tx=0).stop 2024-03-21T08:55:37.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c4c6e700 1 -- 172.21.15.87:0/3248163068 reap_dead start 2024-03-21T08:55:37.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 shutdown_connections 2024-03-21T08:55:37.098 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 >> 172.21.15.87:0/3248163068 conn(0x7fa9c0077fc0 msgr2=0x7fa9c0100410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:37.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 shutdown_connections 2024-03-21T08:55:37.099 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:36.597+0000 7fa9c66d1700 1 -- 172.21.15.87:0/3248163068 wait complete. 2024-03-21T08:55:37.362 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:37.887 INFO:teuthology.orchestra.run.smithi087.stdout:77309411393 2024-03-21T08:55:37.887 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-21T08:55:38.113 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:38 smithi087 ceph-mon[32385]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T08:55:38.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:38 smithi100 ceph-mon[36306]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T08:55:38.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:38 smithi105 ceph-mon[35993]: pgmap v305: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T08:55:40.078 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:40 smithi087 ceph-mon[32385]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:55:40.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:40 smithi100 ceph-mon[36306]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:55:40.285 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:40 smithi105 ceph-mon[35993]: pgmap v306: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:55:42.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:42 smithi100 ceph-mon[36306]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:55:42.287 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:42 smithi087 ceph-mon[32385]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.302+0000 7fb80168a700 1 -- 172.21.15.87:0/1171512979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc1060d0 msgr2=0x7fb7fc1064b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 --2- 172.21.15.87:0/1171512979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc1060d0 0x7fb7fc1064b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009990 tx=0x7fb7ec009ca0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 -- 172.21.15.87:0/1171512979 shutdown_connections 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 --2- 172.21.15.87:0/1171512979 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7fc1025a0 0x7fb7fc10df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 --2- 172.21.15.87:0/1171512979 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc1060d0 0x7fb7fc1064b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 --2- 172.21.15.87:0/1171512979 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc101be0 0x7fb7fc102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 -- 172.21.15.87:0/1171512979 >> 172.21.15.87:0/1171512979 conn(0x7fb7fc0763f0 msgr2=0x7fb7fc076800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 -- 172.21.15.87:0/1171512979 shutdown_connections 2024-03-21T08:55:42.298 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 -- 172.21.15.87:0/1171512979 wait complete. 2024-03-21T08:55:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 Processor -- start 2024-03-21T08:55:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.303+0000 7fb80168a700 1 -- start start 2024-03-21T08:55:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc101be0 0x7fb7fc102dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1025a0 0x7fb7fc103310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.300 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7fc1060d0 0x7fb7fc103850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc1113d0 con 0x7fb7fc101be0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc111250 con 0x7fb7fc1025a0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7fc111550 con 0x7fb7fc1060d0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc101be0 0x7fb7fc102dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc101be0 0x7fb7fc102dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47590/0 (socket says 172.21.15.87:47590) 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 -- 172.21.15.87:0/3096129302 learned_addr learned my addr 172.21.15.87:0/3096129302 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7fa7fc700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1025a0 0x7fb7fc103310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7fb7fe700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7fc1060d0 0x7fb7fc103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 -- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7fc1060d0 msgr2=0x7fb7fc103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb7fc1060d0 0x7fb7fc103850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 -- 172.21.15.87:0/3096129302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1025a0 msgr2=0x7fb7fc103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb7fc1025a0 0x7fb7fc103310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 -- 172.21.15.87:0/3096129302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7fc1a6ee0 con 0x7fb7fc101be0 2024-03-21T08:55:42.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7faffd700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc101be0 0x7fb7fc102dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c00 tx=0x7fb7ec00bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec027070 con 0x7fb7fc101be0 2024-03-21T08:55:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7fc1a7170 con 0x7fb7fc101be0 2024-03-21T08:55:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb7ec003d70 con 0x7fb7fc101be0 2024-03-21T08:55:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7ec00f460 con 0x7fb7fc101be0 2024-03-21T08:55:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.304+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7fc1a75c0 con 0x7fb7fc101be0 2024-03-21T08:55:42.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.305+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7fc0628a0 con 0x7fb7fc101be0 2024-03-21T08:55:42.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.306+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fb7ec02a070 con 0x7fb7fc101be0 2024-03-21T08:55:42.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7e3fff700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb7e40a3040 0x7fb7e40a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.303 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7fa7fc700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb7e40a3040 0x7fb7e40a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fb7ec0d0630 con 0x7fb7fc101be0 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7e3fff700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] conn(0x7fb7e40ac820 0x7fb7e40aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 --> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7ec0043d0 con 0x7fb7e40ac820 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7ec0d0b10 con 0x7fb7fc101be0 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7fa7fc700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb7e40a3040 0x7fb7e40a5500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc076b20 tx=0x7fb7f00073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.307+0000 7fb7fb7fe700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] conn(0x7fb7e40ac820 0x7fb7e40aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.308+0000 7fb7fb7fe700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] conn(0x7fb7e40ac820 0x7fb7e40aec40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:42.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.309+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== osd.11 v2:172.21.15.105:6824/3636544380 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb7ec0043d0 con 0x7fb7e40ac820 2024-03-21T08:55:42.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.325+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 --> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7fc062ca0 con 0x7fb7e40ac820 2024-03-21T08:55:42.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.325+0000 7fb7e3fff700 1 -- 172.21.15.87:0/3096129302 <== osd.11 v2:172.21.15.105:6824/3636544380 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb7fc062ca0 con 0x7fb7e40ac820 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] conn(0x7fb7e40ac820 msgr2=0x7fb7e40aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] conn(0x7fb7e40ac820 0x7fb7e40aec40 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb7e40a3040 msgr2=0x7fb7e40a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb7e40a3040 0x7fb7e40a5500 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc076b20 tx=0x7fb7f00073d0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc101be0 msgr2=0x7fb7fc102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 --2- 172.21.15.87:0/3096129302 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb7fc101be0 0x7fb7fc102dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec000c00 tx=0x7fb7ec00bab0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb7fb7fe700 1 -- 172.21.15.87:0/3096129302 reap_dead start 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 shutdown_connections 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 >> 172.21.15.87:0/3096129302 conn(0x7fb7fc0763f0 msgr2=0x7fb7fc100c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 shutdown_connections 2024-03-21T08:55:42.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.326+0000 7fb80168a700 1 -- 172.21.15.87:0/3096129302 wait complete. 2024-03-21T08:55:42.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:42 smithi105 ceph-mon[35993]: pgmap v307: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:55:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/191822610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4102440 msgr2=0x7fbbb4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/191822610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4102440 0x7fbbb4102820 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbba40099c0 tx=0x7fbba4009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/191822610 shutdown_connections 2024-03-21T08:55:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/191822610 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4107490 0x7fbbb410f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/191822610 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4102440 0x7fbbb4102820 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/191822610 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb4102df0 0x7fbbb4106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/191822610 >> 172.21.15.87:0/191822610 conn(0x7fbbb40792f0 msgr2=0x7fbbb4079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:42.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/191822610 shutdown_connections 2024-03-21T08:55:42.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.529+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/191822610 wait complete. 2024-03-21T08:55:42.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 Processor -- start 2024-03-21T08:55:42.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 -- start start 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102440 0x7fbbb419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb4102df0 0x7fbbb419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4107490 0x7fbbb4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb41124e0 con 0x7fbbb4107490 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb4112360 con 0x7fbbb4102440 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb8f9d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb4112660 con 0x7fbbb4102df0 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102440 0x7fbbb419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbba9d9b700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb4102df0 0x7fbbb419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102440 0x7fbbb419e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:42242/0 (socket says 172.21.15.87:42242) 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb2d9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4107490 0x7fbbb4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 -- 172.21.15.87:0/500136720 learned_addr learned my addr 172.21.15.87:0/500136720 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 -- 172.21.15.87:0/500136720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb4102df0 msgr2=0x7fbbb419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.527 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb4102df0 0x7fbbb419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 -- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4107490 msgr2=0x7fbbb4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbbb4107490 0x7fbbb4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbbb259c700 1 -- 172.21.15.87:0/500136720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4198f60 con 0x7fbbb4102440 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.530+0000 7fbba9d9b700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbbb4102df0 0x7fbbb419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.531+0000 7fbbb259c700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102440 0x7fbbb419e600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbba4006010 tx=0x7fbba4003a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.531+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba4003680 con 0x7fbbb4102440 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.531+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb41991f0 con 0x7fbbb4102440 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.531+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbba40055a0 con 0x7fbbb4102440 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.531+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba400fe00 con 0x7fbbb4102440 2024-03-21T08:55:42.528 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.531+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb41a5460 con 0x7fbbb4102440 2024-03-21T08:55:42.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.532+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbbb4062920 con 0x7fbbb4102440 2024-03-21T08:55:42.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.533+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fbba400f4f0 con 0x7fbbb4102440 2024-03-21T08:55:42.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.533+0000 7fbbabfff700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbba00a31e0 0x7fbba00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.529 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbba9d9b700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbba00a31e0 0x7fbba00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fbba40d01e0 con 0x7fbbb4102440 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbbabfff700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] conn(0x7fbba00ac9c0 0x7fbba00aede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 --> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbba400eea0 con 0x7fbba00ac9c0 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbba40d0590 con 0x7fbbb4102440 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbbb2d9d700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] conn(0x7fbba00ac9c0 0x7fbba00aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbba9d9b700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbba00a31e0 0x7fbba00a56a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4106f70 tx=0x7fbb9c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.534+0000 7fbbb2d9d700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] conn(0x7fbba00ac9c0 0x7fbba00aede0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:42.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.535+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== osd.4 v2:172.21.15.100:6808/3040469961 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fbba400eea0 con 0x7fbba00ac9c0 2024-03-21T08:55:42.546 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.551+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 --> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbbb4062d20 con 0x7fbba00ac9c0 2024-03-21T08:55:42.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbabfff700 1 -- 172.21.15.87:0/500136720 <== osd.4 v2:172.21.15.100:6808/3040469961 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbbb4062d20 con 0x7fbba00ac9c0 2024-03-21T08:55:42.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] conn(0x7fbba00ac9c0 msgr2=0x7fbba00aede0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.547 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] conn(0x7fbba00ac9c0 0x7fbba00aede0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbba00a31e0 msgr2=0x7fbba00a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbba00a31e0 0x7fbba00a56a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbbb4106f70 tx=0x7fbb9c009260 comp rx=0 tx=0).stop 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102440 msgr2=0x7fbbb419e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 --2- 172.21.15.87:0/500136720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbbb4102440 0x7fbbb419e600 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbba4006010 tx=0x7fbba4003a90 comp rx=0 tx=0).stop 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb2d9d700 1 -- 172.21.15.87:0/500136720 reap_dead start 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 shutdown_connections 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 >> 172.21.15.87:0/500136720 conn(0x7fbbb40792f0 msgr2=0x7fbbb4100230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 shutdown_connections 2024-03-21T08:55:42.548 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:42.552+0000 7fbbb8f9d700 1 -- 172.21.15.87:0/500136720 wait complete. 2024-03-21T08:55:44.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:44 smithi100 ceph-mon[36306]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:55:44.268 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:44 smithi087 ceph-mon[32385]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:55:44.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:44 smithi105 ceph-mon[35993]: pgmap v308: 97 pgs: 97 active+clean; 641 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- 172.21.15.87:0/1854237128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 msgr2=0x7f814c101100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- 172.21.15.87:0/1854237128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 0x7f814c101100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f81480099c0 tx=0x7f8148009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- 172.21.15.87:0/1854237128 shutdown_connections 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- 172.21.15.87:0/1854237128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c107540 0x7f814c10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- 172.21.15.87:0/1854237128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 0x7f814c101100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- 172.21.15.87:0/1854237128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c1016d0 0x7f814c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- 172.21.15.87:0/1854237128 >> 172.21.15.87:0/1854237128 conn(0x7f814c0763c0 msgr2=0x7f814c0767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:44.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- 172.21.15.87:0/1854237128 shutdown_connections 2024-03-21T08:55:44.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- 172.21.15.87:0/1854237128 wait complete. 2024-03-21T08:55:44.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 Processor -- start 2024-03-21T08:55:44.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- start start 2024-03-21T08:55:44.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 0x7f814c19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c1016d0 0x7f814c19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c107540 0x7f814c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c1124f0 con 0x7f814c100d20 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c112370 con 0x7f814c107540 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.933+0000 7f8154179700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f814c112670 con 0x7f814c1016d0 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151714700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c1016d0 0x7f814c19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 0x7f814c19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151714700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c1016d0 0x7f814c19eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58466/0 (socket says 172.21.15.87:58466) 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151714700 1 -- 172.21.15.87:0/2088169826 learned_addr learned my addr 172.21.15.87:0/2088169826 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8152716700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c107540 0x7f814c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 -- 172.21.15.87:0/2088169826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c1016d0 msgr2=0x7f814c19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f814c1016d0 0x7f814c19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 -- 172.21.15.87:0/2088169826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c107540 msgr2=0x7f814c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f814c107540 0x7f814c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 -- 172.21.15.87:0/2088169826 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f814c198f70 con 0x7f814c100d20 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8151f15700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 0x7f814c19e610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f81480098e0 tx=0x7f8148005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148027070 con 0x7f814c100d20 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f814c199200 con 0x7f814c100d20 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8148003680 con 0x7f814c100d20 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148022e10 con 0x7f814c100d20 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.934+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f814c1a5470 con 0x7f814c100d20 2024-03-21T08:55:44.932 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.935+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f814c062920 con 0x7f814c100d20 2024-03-21T08:55:44.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.936+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f814802a070 con 0x7f814c100d20 2024-03-21T08:55:44.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8142ffd700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f81380a3040 0x7f81380a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:44.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8151714700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f81380a3040 0x7f81380a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f81480d05e0 con 0x7f814c100d20 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8142ffd700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] conn(0x7f81380ac820 0x7f81380aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 --> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f81380af2f0 con 0x7f81380ac820 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f81480d0ac0 con 0x7f814c100d20 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8152716700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] conn(0x7f81380ac820 0x7f81380aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.937+0000 7f8151714700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f81380a3040 0x7f81380a5500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f814c108950 tx=0x7f813c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.938+0000 7f8152716700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] conn(0x7f81380ac820 0x7f81380aec40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:44.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.938+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== osd.9 v2:172.21.15.87:6826/3198171550 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f81380af2f0 con 0x7f81380ac820 2024-03-21T08:55:44.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 --> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f814c062d20 con 0x7f81380ac820 2024-03-21T08:55:44.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8142ffd700 1 -- 172.21.15.87:0/2088169826 <== osd.9 v2:172.21.15.87:6826/3198171550 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f814c062d20 con 0x7f81380ac820 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] conn(0x7f81380ac820 msgr2=0x7f81380aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] conn(0x7f81380ac820 0x7f81380aec40 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f81380a3040 msgr2=0x7f81380a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f81380a3040 0x7f81380a5500 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f814c108950 tx=0x7f813c009260 comp rx=0 tx=0).stop 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 msgr2=0x7f814c19e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 --2- 172.21.15.87:0/2088169826 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f814c100d20 0x7f814c19e610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f81480098e0 tx=0x7f8148005f10 comp rx=0 tx=0).stop 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8152716700 1 -- 172.21.15.87:0/2088169826 reap_dead start 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 shutdown_connections 2024-03-21T08:55:44.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 >> 172.21.15.87:0/2088169826 conn(0x7f814c0763c0 msgr2=0x7f814c0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:44.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 shutdown_connections 2024-03-21T08:55:44.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:44.955+0000 7f8154179700 1 -- 172.21.15.87:0/2088169826 wait complete. 2024-03-21T08:55:45.432 INFO:teuthology.orchestra.run.smithi087.stdout:313532612632 2024-03-21T08:55:45.433 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.10 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 -- 172.21.15.87:0/2387667255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c107570 msgr2=0x7f8b9c10f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 --2- 172.21.15.87:0/2387667255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c107570 0x7f8b9c10f0f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b0a0 tx=0x7f8b9400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 -- 172.21.15.87:0/2387667255 shutdown_connections 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 --2- 172.21.15.87:0/2387667255 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b9c06aa90 0x7f8b9c06ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 --2- 172.21.15.87:0/2387667255 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c107570 0x7f8b9c10f0f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 --2- 172.21.15.87:0/2387667255 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c06b440 0x7f8b9c106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 -- 172.21.15.87:0/2387667255 >> 172.21.15.87:0/2387667255 conn(0x7f8b9c077f50 msgr2=0x7f8b9c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 -- 172.21.15.87:0/2387667255 shutdown_connections 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 -- 172.21.15.87:0/2387667255 wait complete. 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 Processor -- start 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.092+0000 7f8ba2a55700 1 -- start start 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba2a55700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b9c06aa90 0x7f8b9c1a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba2a55700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c06b440 0x7f8b9c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba2a55700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c107570 0x7f8b9c19a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba2a55700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c112500 con 0x7f8b9c107570 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba2a55700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c112380 con 0x7f8b9c06aa90 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba2a55700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c112680 con 0x7f8b9c06b440 2024-03-21T08:55:46.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c107570 0x7f8b9c19a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c107570 0x7f8b9c19a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:47652/0 (socket says 172.21.15.87:47652) 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 -- 172.21.15.87:0/519668696 learned_addr learned my addr 172.21.15.87:0/519668696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8b9bfff700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b9c06aa90 0x7f8b9c1a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8b9b7fe700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c06b440 0x7f8b9c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 -- 172.21.15.87:0/519668696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c06b440 msgr2=0x7f8b9c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8b9c06b440 0x7f8b9c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 -- 172.21.15.87:0/519668696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b9c06aa90 msgr2=0x7f8b9c1a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8b9c06aa90 0x7f8b9c1a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.091 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.093+0000 7f8ba0ff2700 1 -- 172.21.15.87:0/519668696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c19b000 con 0x7f8b9c107570 2024-03-21T08:55:46.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.094+0000 7f8ba0ff2700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c107570 0x7f8b9c19a8a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b070 tx=0x7f8b94004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:46.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.094+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94021070 con 0x7f8b9c107570 2024-03-21T08:55:46.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.094+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8b94003c00 con 0x7f8b9c107570 2024-03-21T08:55:46.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.094+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b94004630 con 0x7f8b9c107570 2024-03-21T08:55:46.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.094+0000 7f8ba2a55700 1 -- 172.21.15.87:0/519668696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b9c19b290 con 0x7f8b9c107570 2024-03-21T08:55:46.092 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.094+0000 7f8ba2a55700 1 -- 172.21.15.87:0/519668696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b9c1a75b0 con 0x7f8b9c107570 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.095+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b7c001030 con 0x7f8b9c107570 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.097+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8b94007980 con 0x7f8b9c107570 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b997fa700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8b840a30c0 0x7f8b840a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b9bfff700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8b840a30c0 0x7f8b840a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f8b9400f020 con 0x7f8b9c107570 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b997fa700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] conn(0x7f8b840ac770 0x7f8b840aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 --> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b840af240 con 0x7f8b840ac770 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b940d0c90 con 0x7f8b9c107570 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.098+0000 7f8b9b7fe700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] conn(0x7f8b840ac770 0x7f8b840aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.099+0000 7f8b9bfff700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8b840a30c0 0x7f8b840a5580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c19b9e0 tx=0x7f8b8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:46.094 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.099+0000 7f8b9b7fe700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] conn(0x7f8b840ac770 0x7f8b840aeb90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:46.095 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.099+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== osd.3 v2:172.21.15.87:6810/269698093 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8b840af240 con 0x7f8b840ac770 2024-03-21T08:55:46.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 --> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b7c002e60 con 0x7f8b840ac770 2024-03-21T08:55:46.111 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:46 smithi087 ceph-mon[32385]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:55:46.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b997fa700 1 -- 172.21.15.87:0/519668696 <== osd.3 v2:172.21.15.87:6810/269698093 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8b7c002e60 con 0x7f8b840ac770 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] conn(0x7f8b840ac770 msgr2=0x7f8b840aeb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] conn(0x7f8b840ac770 0x7f8b840aeb90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8b840a30c0 msgr2=0x7f8b840a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8b840a30c0 0x7f8b840a5580 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c19b9e0 tx=0x7f8b8c009260 comp rx=0 tx=0).stop 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c107570 msgr2=0x7f8b9c19a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.116+0000 7f8b82ffd700 1 --2- 172.21.15.87:0/519668696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8b9c107570 0x7f8b9c19a8a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8b9400b070 tx=0x7f8b94004c80 comp rx=0 tx=0).stop 2024-03-21T08:55:46.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.117+0000 7f8ba0ff2700 1 -- 172.21.15.87:0/519668696 reap_dead start 2024-03-21T08:55:46.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.117+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 shutdown_connections 2024-03-21T08:55:46.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.117+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 >> 172.21.15.87:0/519668696 conn(0x7f8b9c077f50 msgr2=0x7f8b9c10d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:46.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.117+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 shutdown_connections 2024-03-21T08:55:46.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:46.117+0000 7f8b82ffd700 1 -- 172.21.15.87:0/519668696 wait complete. 2024-03-21T08:55:46.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:46 smithi100 ceph-mon[36306]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:55:46.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:46 smithi105 ceph-mon[35993]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:55:46.777 INFO:teuthology.orchestra.run.smithi087.stdout:292057776160 2024-03-21T08:55:46.777 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.9 2024-03-21T08:55:47.548 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:47.553 INFO:teuthology.orchestra.run.smithi087.stdout:347892350996 2024-03-21T08:55:47.553 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.11 2024-03-21T08:55:47.957 INFO:teuthology.orchestra.run.smithi087.stdout:141733920825 2024-03-21T08:55:47.957 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-21T08:55:48.254 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:48 smithi087 ceph-mon[32385]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-21T08:55:48.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 -- 172.21.15.87:0/1417286691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14069e40 msgr2=0x7f2c1406a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/1417286691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14069e40 0x7f2c1406a2a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f2c000099c0 tx=0x7f2c00009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 -- 172.21.15.87:0/1417286691 shutdown_connections 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/1417286691 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c1406a7e0 0x7f2c1410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/1417286691 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c141081e0 0x7f2c141085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/1417286691 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c14069e40 0x7f2c1406a2a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.258+0000 7f2c1ab52700 1 -- 172.21.15.87:0/1417286691 >> 172.21.15.87:0/1417286691 conn(0x7f2c14077fc0 msgr2=0x7f2c140783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.259+0000 7f2c1ab52700 1 -- 172.21.15.87:0/1417286691 shutdown_connections 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.259+0000 7f2c1ab52700 1 -- 172.21.15.87:0/1417286691 wait complete. 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.259+0000 7f2c1ab52700 1 Processor -- start 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.259+0000 7f2c1ab52700 1 -- start start 2024-03-21T08:55:48.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c1ab52700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c14069e40 0x7f2c1419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c1ab52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c1406a7e0 0x7f2c1419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c1ab52700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c141081e0 0x7f2c141987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c1ab52700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c14112440 con 0x7f2c141081e0 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c1ab52700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141122c0 con 0x7f2c14069e40 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c1ab52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c141125c0 con 0x7f2c1406a7e0 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c14069e40 0x7f2c1419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c190ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c141081e0 0x7f2c141987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c14069e40 0x7f2c1419e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:47904/0 (socket says 172.21.15.87:47904) 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c190ef700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c141081e0 0x7f2c141987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55626/0 (socket says 172.21.15.87:55626) 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 -- 172.21.15.87:0/3118870996 learned_addr learned my addr 172.21.15.87:0/3118870996 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c13fff700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c1406a7e0 0x7f2c1419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:48.264 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 -- 172.21.15.87:0/3118870996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c1406a7e0 msgr2=0x7f2c1419eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c1406a7e0 0x7f2c1419eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 -- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c141081e0 msgr2=0x7f2c141987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c141081e0 0x7f2c141987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 -- 172.21.15.87:0/3118870996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c14198f80 con 0x7f2c14069e40 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c13fff700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2c1406a7e0 0x7f2c1419eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c190ef700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2c141081e0 0x7f2c141987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c188ee700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c14069e40 0x7f2c1419e620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e9a0 tx=0x7f2c0800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.260+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c0800ca10 con 0x7f2c14069e40 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.261+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2c0800cbb0 con 0x7f2c14069e40 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.261+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c08018680 con 0x7f2c14069e40 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.261+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c14199270 con 0x7f2c14069e40 2024-03-21T08:55:48.265 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.261+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c141a5480 con 0x7f2c14069e40 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.261+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2c140628a0 con 0x7f2c14069e40 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.263+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f2c08010430 con 0x7f2c14069e40 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.263+0000 7f2c11ffb700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c040a30c0 0x7f2c040a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.263+0000 7f2c13fff700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c040a30c0 0x7f2c040a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.263+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f2c080cfe50 con 0x7f2c14069e40 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.264+0000 7f2c11ffb700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] conn(0x7f2c040ac8a0 0x7f2c040aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.264+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 --> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2c0801bec0 con 0x7f2c040ac8a0 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.264+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c080d0200 con 0x7f2c14069e40 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.264+0000 7f2c190ef700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] conn(0x7f2c040ac8a0 0x7f2c040aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.264+0000 7f2c13fff700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c040a30c0 0x7f2c040a5580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c14078670 tx=0x7f2c00005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.264+0000 7f2c190ef700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] conn(0x7f2c040ac8a0 0x7f2c040aecc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:48.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.265+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== osd.1 v2:172.21.15.100:6800/4136971528 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2c0801bec0 con 0x7f2c040ac8a0 2024-03-21T08:55:48.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:48 smithi100 ceph-mon[36306]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-21T08:55:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.286+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 --> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f2c14062d70 con 0x7f2c040ac8a0 2024-03-21T08:55:48.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.286+0000 7f2c11ffb700 1 -- 172.21.15.87:0/3118870996 <== osd.1 v2:172.21.15.100:6800/4136971528 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f2c14062d70 con 0x7f2c040ac8a0 2024-03-21T08:55:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.286+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] conn(0x7f2c040ac8a0 msgr2=0x7f2c040aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.286+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] conn(0x7f2c040ac8a0 0x7f2c040aecc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c040a30c0 msgr2=0x7f2c040a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2c040a30c0 0x7f2c040a5580 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2c14078670 tx=0x7f2c00005fb0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.282 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c14069e40 msgr2=0x7f2c1419e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 --2- 172.21.15.87:0/3118870996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2c14069e40 0x7f2c1419e620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2c0800e9a0 tx=0x7f2c0800ecb0 comp rx=0 tx=0).stop 2024-03-21T08:55:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c190ef700 1 -- 172.21.15.87:0/3118870996 reap_dead start 2024-03-21T08:55:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 shutdown_connections 2024-03-21T08:55:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 >> 172.21.15.87:0/3118870996 conn(0x7f2c14077fc0 msgr2=0x7f2c140ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 shutdown_connections 2024-03-21T08:55:48.283 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:48.287+0000 7f2c1ab52700 1 -- 172.21.15.87:0/3118870996 wait complete. 2024-03-21T08:55:48.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:48 smithi105 ceph-mon[35993]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 170 B/s wr, 8 op/s 2024-03-21T08:55:50.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:50 smithi100 ceph-mon[36306]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T08:55:50.267 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:50 smithi087 ceph-mon[32385]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T08:55:50.293 INFO:teuthology.orchestra.run.smithi087.stdout:107374182461 2024-03-21T08:55:50.293 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-21T08:55:50.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:50 smithi105 ceph-mon[35993]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T08:55:50.606 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.608+0000 7fadbc7e9700 1 -- 172.21.15.87:0/1589033624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb4069e40 msgr2=0x7fadb406a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.608+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/1589033624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb4069e40 0x7fadb406a2a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fada40099c0 tx=0x7fada4009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.608+0000 7fadbc7e9700 1 -- 172.21.15.87:0/1589033624 shutdown_connections 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.608+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/1589033624 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb41081e0 0x7fadb41085c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.608+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/1589033624 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb406a7e0 0x7fadb410f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.608+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/1589033624 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb4069e40 0x7fadb406a2a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- 172.21.15.87:0/1589033624 >> 172.21.15.87:0/1589033624 conn(0x7fadb4077fc0 msgr2=0x7fadb40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- 172.21.15.87:0/1589033624 shutdown_connections 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- 172.21.15.87:0/1589033624 wait complete. 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 Processor -- start 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- start start 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb4069e40 0x7fadb419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:50.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb406a7e0 0x7fadb419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb41081e0 0x7fadb4198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4112440 con 0x7fadb41081e0 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb41122c0 con 0x7fadb4069e40 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbc7e9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb41125c0 con 0x7fadb406a7e0 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.609+0000 7fadbad86700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb41081e0 0x7fadb4198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadb9d84700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb406a7e0 0x7fadb419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb41081e0 0x7fadb4198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55646/0 (socket says 172.21.15.87:55646) 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 -- 172.21.15.87:0/4048453491 learned_addr learned my addr 172.21.15.87:0/4048453491 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadba585700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb4069e40 0x7fadb419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 -- 172.21.15.87:0/4048453491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb406a7e0 msgr2=0x7fadb419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadb406a7e0 0x7fadb419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 -- 172.21.15.87:0/4048453491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb4069e40 msgr2=0x7fadb419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadb4069e40 0x7fadb419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 -- 172.21.15.87:0/4048453491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4199000 con 0x7fadb41081e0 2024-03-21T08:55:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbad86700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb41081e0 0x7fadb4198720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fadac007ee0 tx=0x7fadac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac00dd20 con 0x7fadb41081e0 2024-03-21T08:55:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb41992f0 con 0x7fadb41081e0 2024-03-21T08:55:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fadac004020 con 0x7fadb41081e0 2024-03-21T08:55:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac015660 con 0x7fadb41081e0 2024-03-21T08:55:50.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.610+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb41a55d0 con 0x7fadb41081e0 2024-03-21T08:55:50.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.611+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadb40628e0 con 0x7fadb41081e0 2024-03-21T08:55:50.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.612+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fadac0041d0 con 0x7fadb41081e0 2024-03-21T08:55:50.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadab7fe700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fada00a30c0 0x7fada00a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:50.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadba585700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fada00a30c0 0x7fada00a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fadac0cf8c0 con 0x7fadb41081e0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadab7fe700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] conn(0x7fada00ac8a0 0x7fada00aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 --> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadac025e50 con 0x7fada00ac8a0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadac0cfda0 con 0x7fadb41081e0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadb9d84700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] conn(0x7fada00ac8a0 0x7fada00aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.613+0000 7fadba585700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fada00a30c0 0x7fada00a5580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009be0 tx=0x7fadb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.614+0000 7fadb9d84700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] conn(0x7fada00ac8a0 0x7fada00aecc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:50.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.614+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== osd.0 v2:172.21.15.87:6802/3363556070 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fadac025e50 con 0x7fada00ac8a0 2024-03-21T08:55:50.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.631+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 --> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fadb4062d60 con 0x7fada00ac8a0 2024-03-21T08:55:50.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.631+0000 7fadab7fe700 1 -- 172.21.15.87:0/4048453491 <== osd.0 v2:172.21.15.87:6802/3363556070 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fadb4062d60 con 0x7fada00ac8a0 2024-03-21T08:55:50.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] conn(0x7fada00ac8a0 msgr2=0x7fada00aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:50.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] conn(0x7fada00ac8a0 0x7fada00aecc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fada00a30c0 msgr2=0x7fada00a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fada00a30c0 0x7fada00a5580 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fadb0009be0 tx=0x7fadb0009260 comp rx=0 tx=0).stop 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb41081e0 msgr2=0x7fadb4198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 --2- 172.21.15.87:0/4048453491 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadb41081e0 0x7fadb4198720 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fadac007ee0 tx=0x7fadac00d3b0 comp rx=0 tx=0).stop 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbad86700 1 -- 172.21.15.87:0/4048453491 reap_dead start 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 shutdown_connections 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 >> 172.21.15.87:0/4048453491 conn(0x7fadb4077fc0 msgr2=0x7fadb40ffc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.632+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 shutdown_connections 2024-03-21T08:55:50.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:50.633+0000 7fadbc7e9700 1 -- 172.21.15.87:0/4048453491 wait complete. 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 -- 172.21.15.87:0/1446595101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 msgr2=0x7ff4a0102690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1446595101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0102690 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7ff4880099c0 tx=0x7ff488009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 -- 172.21.15.87:0/1446595101 shutdown_connections 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1446595101 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0102bd0 0x7ff4a010f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1446595101 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a0108220 0x7ff4a0108600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1446595101 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0102690 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 -- 172.21.15.87:0/1446595101 >> 172.21.15.87:0/1446595101 conn(0x7ff4a00792f0 msgr2=0x7ff4a0079700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 -- 172.21.15.87:0/1446595101 shutdown_connections 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 -- 172.21.15.87:0/1446595101 wait complete. 2024-03-21T08:55:51.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 Processor -- start 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.041+0000 7ff4a565d700 1 -- start start 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a0102bd0 0x7ff4a00749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0108220 0x7ff4a0074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a01124c0 con 0x7ff4a0102210 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0112340 con 0x7ff4a0108220 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a0112640 con 0x7ff4a0102bd0 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55654/0 (socket says 172.21.15.87:55654) 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 -- 172.21.15.87:0/1714523299 learned_addr learned my addr 172.21.15.87:0/1714523299 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff497fff700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a0102bd0 0x7ff4a00749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49f7fe700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0108220 0x7ff4a0074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:51.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 -- 172.21.15.87:0/1714523299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a0102bd0 msgr2=0x7ff4a00749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff4a0102bd0 0x7ff4a00749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 -- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0108220 msgr2=0x7ff4a0074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff4a0108220 0x7ff4a0074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 -- 172.21.15.87:0/1714523299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a01a6e00 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49effd700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0074490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e9c0 tx=0x7ff49000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49000cbd0 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a01a70f0 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff49000ee60 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff490005630 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.042+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a01a75b0 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.043+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff4a0062920 con 0x7ff4a0102210 2024-03-21T08:55:51.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.044+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7ff490004750 con 0x7ff4a0102210 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff49cff9700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff48c0a3040 0x7ff48c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff497fff700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff48c0a3040 0x7ff48c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff4900cf9f0 con 0x7ff4a0102210 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff49cff9700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] conn(0x7ff48c0ac820 0x7ff48c0aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 --> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff48c0af2f0 con 0x7ff48c0ac820 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff4900cfed0 con 0x7ff4a0102210 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.045+0000 7ff497fff700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff48c0a3040 0x7ff48c0a5500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0075f20 tx=0x7ff488005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.046+0000 7ff49f7fe700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] conn(0x7ff48c0ac820 0x7ff48c0aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.046+0000 7ff49f7fe700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] conn(0x7ff48c0ac820 0x7ff48c0aec40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:51.059 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.047+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== osd.7 v2:172.21.15.100:6816/1198353584 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff48c0af2f0 con 0x7ff48c0ac820 2024-03-21T08:55:51.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.063+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 --> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7ff4a0062da0 con 0x7ff48c0ac820 2024-03-21T08:55:51.060 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.063+0000 7ff49cff9700 1 -- 172.21.15.87:0/1714523299 <== osd.7 v2:172.21.15.100:6816/1198353584 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7ff4a0062da0 con 0x7ff48c0ac820 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] conn(0x7ff48c0ac820 msgr2=0x7ff48c0aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] conn(0x7ff48c0ac820 0x7ff48c0aec40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff48c0a3040 msgr2=0x7ff48c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff48c0a3040 0x7ff48c0a5500 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7ff4a0075f20 tx=0x7ff488005fb0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 msgr2=0x7ff4a0074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 --2- 172.21.15.87:0/1714523299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff4a0102210 0x7ff4a0074490 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7ff49000e9c0 tx=0x7ff49000ecd0 comp rx=0 tx=0).stop 2024-03-21T08:55:51.061 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff49f7fe700 1 -- 172.21.15.87:0/1714523299 reap_dead start 2024-03-21T08:55:51.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 shutdown_connections 2024-03-21T08:55:51.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 >> 172.21.15.87:0/1714523299 conn(0x7ff4a00792f0 msgr2=0x7ff4a0100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:51.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 shutdown_connections 2024-03-21T08:55:51.062 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:51.064+0000 7ff4a565d700 1 -- 172.21.15.87:0/1714523299 wait complete. 2024-03-21T08:55:52.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:52 smithi087 ceph-mon[32385]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T08:55:52.307 INFO:teuthology.orchestra.run.smithi087.stdout:223338299435 2024-03-21T08:55:52.307 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-21T08:55:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 -- 172.21.15.87:0/1047842894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c1063a0 msgr2=0x7fd46c10df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 --2- 172.21.15.87:0/1047842894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c1063a0 0x7fd46c10df70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd46400b0d0 tx=0x7fd46400b3e0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 -- 172.21.15.87:0/1047842894 shutdown_connections 2024-03-21T08:55:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 --2- 172.21.15.87:0/1047842894 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c101350 0x7fd46c101730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 --2- 172.21.15.87:0/1047842894 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c1063a0 0x7fd46c10df70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 --2- 172.21.15.87:0/1047842894 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd46c101d00 0x7fd46c105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.486+0000 7fd4715fa700 1 -- 172.21.15.87:0/1047842894 >> 172.21.15.87:0/1047842894 conn(0x7fd46c0770e0 msgr2=0x7fd46c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.487+0000 7fd4715fa700 1 -- 172.21.15.87:0/1047842894 shutdown_connections 2024-03-21T08:55:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.487+0000 7fd4715fa700 1 -- 172.21.15.87:0/1047842894 wait complete. 2024-03-21T08:55:52.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.487+0000 7fd4715fa700 1 Processor -- start 2024-03-21T08:55:52.485 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:52 smithi105 ceph-mon[35993]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.487+0000 7fd4715fa700 1 -- start start 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd4715fa700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c101350 0x7fd46c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd4715fa700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c101d00 0x7fd46c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd4715fa700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd46c1063a0 0x7fd46c1975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c101d00 0x7fd46c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd4715fa700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c1113b0 con 0x7fd46c101d00 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c101d00 0x7fd46c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55674/0 (socket says 172.21.15.87:55674) 2024-03-21T08:55:52.487 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 -- 172.21.15.87:0/2141071513 learned_addr learned my addr 172.21.15.87:0/2141071513 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd4715fa700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c111230 con 0x7fd46c101350 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46c111530 con 0x7fd46c1063a0 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46affd700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c101350 0x7fd46c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46b7fe700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd46c1063a0 0x7fd46c1975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 -- 172.21.15.87:0/2141071513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd46c1063a0 msgr2=0x7fd46c1975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd46c1063a0 0x7fd46c1975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 -- 172.21.15.87:0/2141071513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c101350 msgr2=0x7fd46c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd46c101350 0x7fd46c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.488+0000 7fd46a7fc700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd46c197d50 con 0x7fd46c101d00 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.489+0000 7fd46a7fc700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c101d00 0x7fd46c19d970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e9f0 tx=0x7fd46000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:52.488 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.489+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460004a30 con 0x7fd46c101d00 2024-03-21T08:55:52.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.489+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd46000cca0 con 0x7fd46c101d00 2024-03-21T08:55:52.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.489+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600056c0 con 0x7fd46c101d00 2024-03-21T08:55:52.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.489+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd46c198040 con 0x7fd46c101d00 2024-03-21T08:55:52.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.489+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd46c06a890 con 0x7fd46c101d00 2024-03-21T08:55:52.489 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.490+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd46c0ffac0 con 0x7fd46c101d00 2024-03-21T08:55:52.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.491+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fd460004d60 con 0x7fd46c101d00 2024-03-21T08:55:52.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.492+0000 7fd453fff700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd4540a30c0 0x7fd4540a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.492+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd4600cf8f0 con 0x7fd46c101d00 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.492+0000 7fd46affd700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd4540a30c0 0x7fd4540a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.492+0000 7fd453fff700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] conn(0x7fd4540ac8a0 0x7fd4540aecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.492+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd460012ec0 con 0x7fd4540ac8a0 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.492+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4600cfdd0 con 0x7fd46c101d00 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.493+0000 7fd46b7fe700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] conn(0x7fd4540ac8a0 0x7fd4540aecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.493+0000 7fd46affd700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd4540a30c0 0x7fd4540a5580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009c60 tx=0x7fd45c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.493+0000 7fd46b7fe700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] conn(0x7fd4540ac8a0 0x7fd4540aecc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:52.491 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.494+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== osd.6 v2:172.21.15.87:6818/944234078 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd460012ec0 con 0x7fd4540ac8a0 2024-03-21T08:55:52.500 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:52.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:52 smithi100 ceph-mon[36306]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 170 B/s wr, 9 op/s 2024-03-21T08:55:52.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.526+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 --> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd46c1027a0 con 0x7fd4540ac8a0 2024-03-21T08:55:52.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.527+0000 7fd453fff700 1 -- 172.21.15.87:0/2141071513 <== osd.6 v2:172.21.15.87:6818/944234078 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd46c1027a0 con 0x7fd4540ac8a0 2024-03-21T08:55:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] conn(0x7fd4540ac8a0 msgr2=0x7fd4540aecc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] conn(0x7fd4540ac8a0 0x7fd4540aecc0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:52.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd4540a30c0 msgr2=0x7fd4540a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd4540a30c0 0x7fd4540a5580 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fd45c009c60 tx=0x7fd45c009260 comp rx=0 tx=0).stop 2024-03-21T08:55:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c101d00 msgr2=0x7fd46c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 --2- 172.21.15.87:0/2141071513 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd46c101d00 0x7fd46c19d970 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd46000e9f0 tx=0x7fd46000ed00 comp rx=0 tx=0).stop 2024-03-21T08:55:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd46b7fe700 1 -- 172.21.15.87:0/2141071513 reap_dead start 2024-03-21T08:55:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 shutdown_connections 2024-03-21T08:55:52.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.528+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 >> 172.21.15.87:0/2141071513 conn(0x7fd46c0770e0 msgr2=0x7fd46c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.530+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 shutdown_connections 2024-03-21T08:55:52.526 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:52.530+0000 7fd4715fa700 1 -- 172.21.15.87:0/2141071513 wait complete. 2024-03-21T08:55:52.914 INFO:teuthology.orchestra.run.smithi087.stdout:34359738446 2024-03-21T08:55:52.915 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-21T08:55:53.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 -- 172.21.15.87:0/3770188076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981081c0 msgr2=0x7fc9981085a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:53.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 --2- 172.21.15.87:0/3770188076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981081c0 0x7fc9981085a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9880099c0 tx=0x7fc988009cd0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 -- 172.21.15.87:0/3770188076 shutdown_connections 2024-03-21T08:55:53.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 --2- 172.21.15.87:0/3770188076 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc99806b2a0 0x7fc99810efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 --2- 172.21.15.87:0/3770188076 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc9981081c0 0x7fc9981085a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 --2- 172.21.15.87:0/3770188076 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99806a8e0 0x7fc99806ad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 -- 172.21.15.87:0/3770188076 >> 172.21.15.87:0/3770188076 conn(0x7fc9980780a0 msgr2=0x7fc9980784b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 -- 172.21.15.87:0/3770188076 shutdown_connections 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.138+0000 7fc99cd29700 1 -- 172.21.15.87:0/3770188076 wait complete. 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 Processor -- start 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 -- start start 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99806a8e0 0x7fc998101140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99806b2a0 0x7fc998101680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9981081c0 0x7fc998101bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc998112470 con 0x7fc99806b2a0 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9981122f0 con 0x7fc99806a8e0 2024-03-21T08:55:53.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.139+0000 7fc99cd29700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9981125f0 con 0x7fc9981081c0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99806b2a0 0x7fc998101680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99806b2a0 0x7fc998101680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:55690/0 (socket says 172.21.15.87:55690) 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 -- 172.21.15.87:0/2966994865 learned_addr learned my addr 172.21.15.87:0/2966994865 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc99659c700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99806a8e0 0x7fc998101140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc996d9d700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9981081c0 0x7fc998101bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 -- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9981081c0 msgr2=0x7fc998101bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc9981081c0 0x7fc998101bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 -- 172.21.15.87:0/2966994865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99806a8e0 msgr2=0x7fc998101140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc99806a8e0 0x7fc998101140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 -- 172.21.15.87:0/2966994865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9981a6e10 con 0x7fc99806b2a0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc995d9b700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99806b2a0 0x7fc998101680 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9a0 tx=0x7fc98c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c00ca10 con 0x7fc99806b2a0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc98c00cbb0 con 0x7fc99806b2a0 2024-03-21T08:55:53.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9981a7100 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98c018680 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.140+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9981a7640 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.141+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc998062860 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.142+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fc98c010430 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc9877fe700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc9800a3040 0x7fc9800a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc99659c700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc9800a3040 0x7fc9800a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc98c0cfe20 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc9877fe700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] conn(0x7fc9800ac820 0x7fc9800aec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 --> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc98c01bec0 con 0x7fc9800ac820 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_get_version_reply(handle=1 version=86) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc98c0d0300 con 0x7fc99806b2a0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc996d9d700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] conn(0x7fc9800ac820 0x7fc9800aec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:53.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.143+0000 7fc99659c700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc9800a3040 0x7fc9800a5500 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc988000c00 tx=0x7fc988005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.144+0000 7fc996d9d700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] conn(0x7fc9800ac820 0x7fc9800aec40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.144+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== osd.8 v2:172.21.15.105:6816/2122777536 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc98c01bec0 con 0x7fc9800ac820 2024-03-21T08:55:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 --> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc998062ce0 con 0x7fc9800ac820 2024-03-21T08:55:53.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc9877fe700 1 -- 172.21.15.87:0/2966994865 <== osd.8 v2:172.21.15.105:6816/2122777536 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc998062ce0 con 0x7fc9800ac820 2024-03-21T08:55:53.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] conn(0x7fc9800ac820 msgr2=0x7fc9800aec40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:53.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] conn(0x7fc9800ac820 0x7fc9800aec40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc9800a3040 msgr2=0x7fc9800a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc9800a3040 0x7fc9800a5500 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc988000c00 tx=0x7fc988005fb0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99806b2a0 msgr2=0x7fc998101680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 --2- 172.21.15.87:0/2966994865 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc99806b2a0 0x7fc998101680 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc98c00e9a0 tx=0x7fc98c00ecb0 comp rx=0 tx=0).stop 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc996d9d700 1 -- 172.21.15.87:0/2966994865 reap_dead start 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.162+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 shutdown_connections 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.163+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 >> 172.21.15.87:0/2966994865 conn(0x7fc9980780a0 msgr2=0x7fc9980ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.163+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 shutdown_connections 2024-03-21T08:55:53.167 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:53.163+0000 7fc99cd29700 1 -- 172.21.15.87:0/2966994865 wait complete. 2024-03-21T08:55:53.802 INFO:teuthology.orchestra.run.smithi087.stdout:201863462960 2024-03-21T08:55:53.802 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-21T08:55:54.009 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:54.019 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:54.043 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:54 smithi087 ceph-mon[32385]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:55:54.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:54 smithi105 ceph-mon[35993]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:55:54.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:54 smithi100 ceph-mon[36306]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:55:56.390 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:56 smithi087 ceph-mon[32385]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:55:56.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:56 smithi105 ceph-mon[35993]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:55:56.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:56 smithi100 ceph-mon[36306]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:55:56.683 INFO:teuthology.orchestra.run.smithi087.stdout:257698037798 2024-03-21T08:55:56.683 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.8 2024-03-21T08:55:57.329 INFO:teuthology.orchestra.run.smithi087.stdout:55834574921 2024-03-21T08:55:57.329 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 -- 172.21.15.87:0/4145566945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 msgr2=0x7f77dc10f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 --2- 172.21.15.87:0/4145566945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc10f030 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f77d400b0a0 tx=0x7f77d400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 -- 172.21.15.87:0/4145566945 shutdown_connections 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 --2- 172.21.15.87:0/4145566945 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc10f030 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 --2- 172.21.15.87:0/4145566945 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc1082f0 0x7f77dc1086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 --2- 172.21.15.87:0/4145566945 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc102210 0x7f77dc102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 -- 172.21.15.87:0/4145566945 >> 172.21.15.87:0/4145566945 conn(0x7f77dc079250 msgr2=0x7f77dc079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 -- 172.21.15.87:0/4145566945 shutdown_connections 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.521+0000 7f77e11b2700 1 -- 172.21.15.87:0/4145566945 wait complete. 2024-03-21T08:55:57.518 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.522+0000 7f77e11b2700 1 Processor -- start 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.522+0000 7f77e11b2700 1 -- start start 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.522+0000 7f77e11b2700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc102210 0x7f77dc19e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.522+0000 7f77e11b2700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc19eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77e11b2700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc1082f0 0x7f77dc198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77e11b2700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc112450 con 0x7f77dc102bd0 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77e11b2700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc1122d0 con 0x7f77dc1082f0 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77e11b2700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc1125d0 con 0x7f77dc102210 2024-03-21T08:55:57.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77da59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc19eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77dad9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc102210 0x7f77dc19e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc1082f0 0x7f77dc198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77da59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc19eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37860/0 (socket says 172.21.15.87:37860) 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77da59c700 1 -- 172.21.15.87:0/1913383500 learned_addr learned my addr 172.21.15.87:0/1913383500 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 -- 172.21.15.87:0/1913383500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc102210 msgr2=0x7f77dc19e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc102210 0x7f77dc19e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 -- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 msgr2=0x7f77dc19eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc19eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 -- 172.21.15.87:0/1913383500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc198f70 con 0x7f77dc1082f0 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77da59c700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77db59e700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc1082f0 0x7f77dc198690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77d400b0a0 tx=0x7f77d4004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:57.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.523+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4021070 con 0x7f77dc1082f0 2024-03-21T08:55:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.524+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc199200 con 0x7f77dc1082f0 2024-03-21T08:55:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.524+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc1a5540 con 0x7f77dc1082f0 2024-03-21T08:55:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.524+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f77d4007b80 con 0x7f77dc1082f0 2024-03-21T08:55:57.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.524+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77d4005040 con 0x7f77dc1082f0 2024-03-21T08:55:57.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.525+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77bc005330 con 0x7f77dc1082f0 2024-03-21T08:55:57.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.526+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f77d4024070 con 0x7f77dc1082f0 2024-03-21T08:55:57.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.526+0000 7f77d3fff700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a30c0 0x7f77c80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:55:57.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.527+0000 7f77dad9d700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a30c0 0x7f77c80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:55:57.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.527+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f77d400f020 con 0x7f77dc1082f0 2024-03-21T08:55:57.524 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.527+0000 7f77dad9d700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a30c0 0x7f77c80a5580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009be0 tx=0x7f77cc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:55:57.530 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.535+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f77d402b020 con 0x7f77dc1082f0 2024-03-21T08:55:57.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.738+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f77bc0058f0 con 0x7f77dc1082f0 2024-03-21T08:55:57.735 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.739+0000 7f77d3fff700 1 -- 172.21.15.87:0/1913383500 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f77d409d880 con 0x7f77dc1082f0 2024-03-21T08:55:57.735 INFO:teuthology.orchestra.run.smithi087.stdout:77309411397 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a30c0 msgr2=0x7f77c80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a30c0 0x7f77c80a5580 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f77cc009be0 tx=0x7f77cc009260 comp rx=0 tx=0).stop 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc1082f0 msgr2=0x7f77dc198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc1082f0 0x7f77dc198690 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f77d400b0a0 tx=0x7f77d4004d00 comp rx=0 tx=0).stop 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 shutdown_connections 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f77c80a30c0 0x7f77c80a5580 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f77dc1082f0 0x7f77dc198690 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.737 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f77dc102210 0x7f77dc19e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 --2- 172.21.15.87:0/1913383500 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f77dc102bd0 0x7f77dc19eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:55:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 >> 172.21.15.87:0/1913383500 conn(0x7f77dc079250 msgr2=0x7f77dc100d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:55:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 shutdown_connections 2024-03-21T08:55:57.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:55:57.741+0000 7f77e11b2700 1 -- 172.21.15.87:0/1913383500 wait complete. 2024-03-21T08:55:58.210 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:55:58.391 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:58 smithi087 ceph-mon[32385]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:55:58.391 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:58 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1913383500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T08:55:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:58 smithi105 ceph-mon[35993]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:55:58.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:58 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1913383500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T08:55:58.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:58 smithi100 ceph-mon[36306]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:55:58.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:58 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1913383500' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-21T08:55:59.052 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411393 got 77309411397 for osd.2 2024-03-21T08:55:59.053 DEBUG:teuthology.parallel:result is None 2024-03-21T08:55:59.281 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-21T08:55:59.281 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-21T08:55:59.281 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-21T08:55:59.282 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-21T08:55:59.282 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-21T08:55:59.282 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-21T08:55:59.282 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:55:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-03-21T08:55:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-21T08:55:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-21T08:55:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-21T08:55:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-21T08:55:59.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-21T08:55:59.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-21T08:55:59.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:55:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-03-21T08:55:59.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]: dispatch 2024-03-21T08:55:59.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-03-21T08:55:59.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]: dispatch 2024-03-21T08:55:59.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]: dispatch 2024-03-21T08:55:59.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]: dispatch 2024-03-21T08:55:59.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-03-21T08:55:59.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:55:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]: dispatch 2024-03-21T08:56:00.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:00.152 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-03-21T08:56:00.153 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:00 smithi087 ceph-mon[32385]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T08:56:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-03-21T08:56:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:00 smithi105 ceph-mon[35993]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T08:56:00.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: pgmap v316: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:00.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.8", "id": [2, 11]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.13", "id": [4, 9]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.14", "id": [4, 6]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.18", "id": [3, 9]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.37", "id": [5, 8]}]': finished 2024-03-21T08:56:00.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:00 smithi100 ceph-mon[36306]: osdmap e87: 12 total, 12 up, 12 in 2024-03-21T08:56:01.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:01 smithi087 ceph-mon[32385]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T08:56:01.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:01 smithi087 ceph-mon[32385]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-03-21T08:56:01.204 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:01.205 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:01.208 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:01.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:01 smithi105 ceph-mon[35993]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T08:56:01.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:01 smithi105 ceph-mon[35993]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-03-21T08:56:01.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:01 smithi100 ceph-mon[36306]: osdmap e88: 12 total, 12 up, 12 in 2024-03-21T08:56:01.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:01 smithi100 ceph-mon[36306]: pgmap v319: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-03-21T08:56:01.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.910+0000 7f28f3a52700 1 -- 172.21.15.87:0/1093424836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec1082f0 msgr2=0x7f28ec1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.910+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1093424836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec1082f0 0x7f28ec1086d0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009990 tx=0x7f28e8009ca0 comp rx=0 tx=0).stop 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- 172.21.15.87:0/1093424836 shutdown_connections 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1093424836 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec102bd0 0x7f28ec10f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1093424836 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec1082f0 0x7f28ec1086d0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1093424836 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ec102210 0x7f28ec102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- 172.21.15.87:0/1093424836 >> 172.21.15.87:0/1093424836 conn(0x7f28ec079250 msgr2=0x7f28ec079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- 172.21.15.87:0/1093424836 shutdown_connections 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- 172.21.15.87:0/1093424836 wait complete. 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 Processor -- start 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- start start 2024-03-21T08:56:01.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ec102210 0x7f28ec19e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 0x7f28ec19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec1082f0 0x7f28ec198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec112450 con 0x7f28ec102bd0 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec1122d0 con 0x7f28ec1082f0 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.911+0000 7f28f3a52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28ec1125d0 con 0x7f28ec102210 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 0x7f28ec19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 0x7f28ec19ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37870/0 (socket says 172.21.15.87:37870) 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 -- 172.21.15.87:0/1982893762 learned_addr learned my addr 172.21.15.87:0/1982893762 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f17ee700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ec102210 0x7f28ec19e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f1fef700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec1082f0 0x7f28ec198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 -- 172.21.15.87:0/1982893762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ec102210 msgr2=0x7f28ec19e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:01.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ec102210 0x7f28ec19e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 -- 172.21.15.87:0/1982893762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec1082f0 msgr2=0x7f28ec198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec1082f0 0x7f28ec198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 -- 172.21.15.87:0/1982893762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28ec199070 con 0x7f28ec102bd0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f0fed700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 0x7f28ec19ebb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9a0 tx=0x7f28dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc00ca10 con 0x7f28ec102bd0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f28dc00cbb0 con 0x7f28ec102bd0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec199360 con 0x7f28ec102bd0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28dc018680 con 0x7f28ec102bd0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.912+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28ec1a55f0 con 0x7f28ec102bd0 2024-03-21T08:56:01.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.913+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28ec1038d0 con 0x7f28ec102bd0 2024-03-21T08:56:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.914+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f28dc010430 con 0x7f28ec102bd0 2024-03-21T08:56:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.915+0000 7f28e27fc700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f28d80a30c0 0x7f28d80a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.915+0000 7f28f17ee700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f28d80a30c0 0x7f28d80a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.915+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f28dc0cfb90 con 0x7f28ec102bd0 2024-03-21T08:56:01.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.915+0000 7f28f17ee700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f28d80a30c0 0x7f28d80a5580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b2e0 tx=0x7f28e8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:01.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:01.918+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f28dc09c890 con 0x7f28ec102bd0 2024-03-21T08:56:01.930 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:01.930 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:01.931 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:02.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.136+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f28ec199c70 con 0x7f28ec102bd0 2024-03-21T08:56:02.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.137+0000 7f28e27fc700 1 -- 172.21.15.87:0/1982893762 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f28dc09c190 con 0x7f28ec102bd0 2024-03-21T08:56:02.133 INFO:teuthology.orchestra.run.smithi087.stdout:347892350999 2024-03-21T08:56:02.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f28d80a30c0 msgr2=0x7f28d80a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f28d80a30c0 0x7f28d80a5580 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f28e800b2e0 tx=0x7f28e8005960 comp rx=0 tx=0).stop 2024-03-21T08:56:02.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 msgr2=0x7f28ec19ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.135 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 0x7f28ec19ebb0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f28dc00e9a0 tx=0x7f28dc00ecb0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 shutdown_connections 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f28d80a30c0 0x7f28d80a5580 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f28ec1082f0 0x7f28ec198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f28ec102210 0x7f28ec19e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 --2- 172.21.15.87:0/1982893762 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f28ec102bd0 0x7f28ec19ebb0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 >> 172.21.15.87:0/1982893762 conn(0x7f28ec079250 msgr2=0x7f28ec100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 shutdown_connections 2024-03-21T08:56:02.136 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.139+0000 7f28f3a52700 1 -- 172.21.15.87:0/1982893762 wait complete. 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 -- 172.21.15.87:0/4038093450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f006b2e0 msgr2=0x7f67f0106de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 --2- 172.21.15.87:0/4038093450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f006b2e0 0x7f67f0106de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f67d80099c0 tx=0x7f67d8009cd0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 -- 172.21.15.87:0/4038093450 shutdown_connections 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 --2- 172.21.15.87:0/4038093450 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f0107510 0x7f67f010f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 --2- 172.21.15.87:0/4038093450 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006a930 0x7f67f006ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 --2- 172.21.15.87:0/4038093450 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f006b2e0 0x7f67f0106de0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 -- 172.21.15.87:0/4038093450 >> 172.21.15.87:0/4038093450 conn(0x7f67f00770e0 msgr2=0x7f67f00774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 -- 172.21.15.87:0/4038093450 shutdown_connections 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.152+0000 7f67f5c52700 1 -- 172.21.15.87:0/4038093450 wait complete. 2024-03-21T08:56:02.149 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 Processor -- start 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 -- start start 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f006a930 0x7f67f019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 0x7f67f019eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f0107510 0x7f67f0198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f0112460 con 0x7f67f0107510 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01122e0 con 0x7f67f006a930 2024-03-21T08:56:02.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67f5c52700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f01125e0 con 0x7f67f006b2e0 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.153+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 0x7f67f019eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67effff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f0107510 0x7f67f0198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 0x7f67f019eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:52024/0 (socket says 172.21.15.87:52024) 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 -- 172.21.15.87:0/2310669737 learned_addr learned my addr 172.21.15.87:0/2310669737 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67ef7fe700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f006a930 0x7f67f019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 -- 172.21.15.87:0/2310669737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f006a930 msgr2=0x7f67f019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f006a930 0x7f67f019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 -- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f0107510 msgr2=0x7f67f0198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f0107510 0x7f67f0198820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 -- 172.21.15.87:0/2310669737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67f0199000 con 0x7f67f006b2e0 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67ef7fe700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f006a930 0x7f67f019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67eeffd700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 0x7f67f019eb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b580 tx=0x7f67d8003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:02.156 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d8003680 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f67d80055a0 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f0199290 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d800fd00 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.154+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f01a5460 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.155+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f01038c0 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.157+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f67d800f460 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.157+0000 7f67ecff9700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67dc0a3260 0x7f67dc0a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.158+0000 7f67ef7fe700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67dc0a3260 0x7f67dc0a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.158+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67d80d0760 con 0x7f67f006b2e0 2024-03-21T08:56:02.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.158+0000 7f67ef7fe700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67dc0a3260 0x7f67dc0a5720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009480 tx=0x7f67e800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:02.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.161+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f67d8013070 con 0x7f67f006b2e0 2024-03-21T08:56:02.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.373+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f67f0199cb0 con 0x7f67f006b2e0 2024-03-21T08:56:02.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.374+0000 7f67ecff9700 1 -- 172.21.15.87:0/2310669737 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f67d809d490 con 0x7f67f006b2e0 2024-03-21T08:56:02.370 INFO:teuthology.orchestra.run.smithi087.stdout:107374182464 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67dc0a3260 msgr2=0x7f67dc0a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67dc0a3260 0x7f67dc0a5720 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009480 tx=0x7f67e800b040 comp rx=0 tx=0).stop 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 msgr2=0x7f67f019eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 0x7f67f019eb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67d800b580 tx=0x7f67d8003a90 comp rx=0 tx=0).stop 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 shutdown_connections 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67dc0a3260 0x7f67dc0a5720 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f67f006a930 0x7f67f019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f67f0107510 0x7f67f0198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 --2- 172.21.15.87:0/2310669737 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f67f006b2e0 0x7f67f019eb40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.376+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 >> 172.21.15.87:0/2310669737 conn(0x7f67f00770e0 msgr2=0x7f67f00ff530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.377+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 shutdown_connections 2024-03-21T08:56:02.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:02.377+0000 7f67f5c52700 1 -- 172.21.15.87:0/2310669737 wait complete. 2024-03-21T08:56:02.383 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:02 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1982893762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T08:56:02.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:02 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1982893762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T08:56:02.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:02 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1982893762' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-03-21T08:56:03.046 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 -- 172.21.15.87:0/3089403514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 msgr2=0x7fa2b41074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 --2- 172.21.15.87:0/3089403514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b41074b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fa2a8009990 tx=0x7fa2a8009ca0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 -- 172.21.15.87:0/3089403514 shutdown_connections 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 --2- 172.21.15.87:0/3089403514 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b4101a30 0x7fa2b410df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 --2- 172.21.15.87:0/3089403514 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b41074b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 --2- 172.21.15.87:0/3089403514 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b4101090 0x7fa2b41014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 -- 172.21.15.87:0/3089403514 >> 172.21.15.87:0/3089403514 conn(0x7fa2b4077fc0 msgr2=0x7fa2b40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 -- 172.21.15.87:0/3089403514 shutdown_connections 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 -- 172.21.15.87:0/3089403514 wait complete. 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.050+0000 7fa2b9517700 1 Processor -- start 2024-03-21T08:56:03.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 -- start start 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b4101090 0x7fa2b419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b4101a30 0x7fa2b419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b4111330 con 0x7fa2b41070d0 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41111b0 con 0x7fa2b4101090 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b9517700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2b41114b0 con 0x7fa2b4101a30 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b4198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:37906/0 (socket says 172.21.15.87:37906) 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 -- 172.21.15.87:0/686382768 learned_addr learned my addr 172.21.15.87:0/686382768 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:03.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b2ffd700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b4101090 0x7fa2b419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b27fc700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b4101a30 0x7fa2b419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 -- 172.21.15.87:0/686382768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b4101a30 msgr2=0x7fa2b419eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b4101a30 0x7fa2b419eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 -- 172.21.15.87:0/686382768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b4101090 msgr2=0x7fa2b419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b4101090 0x7fa2b419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 -- 172.21.15.87:0/686382768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b4198f60 con 0x7fa2b41070d0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.051+0000 7fa2b37fe700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b4198780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00ef30 tx=0x7fa2ac00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.052+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac0092e0 con 0x7fa2b41070d0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.052+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2b41991f0 con 0x7fa2b41070d0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.052+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2ac004bb0 con 0x7fa2b41070d0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.052+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ac004280 con 0x7fa2b41070d0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.052+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2b41a5460 con 0x7fa2b41070d0 2024-03-21T08:56:03.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.053+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2b4102780 con 0x7fa2b41070d0 2024-03-21T08:56:03.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.054+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa2ac01b030 con 0x7fa2b41070d0 2024-03-21T08:56:03.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.054+0000 7fa29bfff700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa29c0a3040 0x7fa29c0a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:03.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.054+0000 7fa2b2ffd700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa29c0a3040 0x7fa29c0a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:03.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.054+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa2ac0cf970 con 0x7fa2b41070d0 2024-03-21T08:56:03.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.055+0000 7fa2b2ffd700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa29c0a3040 0x7fa29c0a5500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800b2e0 tx=0x7fa2a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:03.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.058+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2ac022020 con 0x7fa2b41070d0 2024-03-21T08:56:03.257 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.261+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa2b41a5630 con 0x7fa2b41070d0 2024-03-21T08:56:03.257 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2310669737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T08:56:03.257 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:03 smithi087 ceph-mon[32385]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:03.258 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.261+0000 7fa29bfff700 1 -- 172.21.15.87:0/686382768 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa2ac010040 con 0x7fa2b41070d0 2024-03-21T08:56:03.258 INFO:teuthology.orchestra.run.smithi087.stdout:167503724600 2024-03-21T08:56:03.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa29c0a3040 msgr2=0x7fa29c0a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:03.259 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa29c0a3040 0x7fa29c0a5500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa2a800b2e0 tx=0x7fa2a8005960 comp rx=0 tx=0).stop 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 msgr2=0x7fa2b4198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b4198780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa2ac00ef30 tx=0x7fa2ac00c3a0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 shutdown_connections 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa29c0a3040 0x7fa29c0a5500 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2b4101090 0x7fa2b419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2b41070d0 0x7fa2b4198780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 --2- 172.21.15.87:0/686382768 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2b4101a30 0x7fa2b419eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.263+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 >> 172.21.15.87:0/686382768 conn(0x7fa2b4077fc0 msgr2=0x7fa2b4100420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.264+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 shutdown_connections 2024-03-21T08:56:03.260 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:03.264+0000 7fa2b9517700 1 -- 172.21.15.87:0/686382768 wait complete. 2024-03-21T08:56:03.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2310669737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T08:56:03.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:03 smithi105 ceph-mon[35993]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:03.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2310669737' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-21T08:56:03.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:03 smithi100 ceph-mon[36306]: pgmap v320: 97 pgs: 1 peering, 96 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:03.546 INFO:tasks.cephadm.ceph_manager.ceph:need seq 347892350996 got 347892350999 for osd.11 2024-03-21T08:56:03.546 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:04.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:04 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/686382768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T08:56:04.511 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:04 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/686382768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T08:56:04.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:04 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/686382768' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-21T08:56:05.249 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:56:05.249 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:05 smithi087 ceph-mon[32385]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:05.280 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182461 got 107374182464 for osd.3 2024-03-21T08:56:05.280 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:05.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:56:05.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:05 smithi105 ceph-mon[35993]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:05.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:56:05.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:05 smithi100 ceph-mon[36306]: pgmap v321: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 12 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:06.268 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724593 got 167503724600 for osd.5 2024-03-21T08:56:06.268 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:07.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.642 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.643 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:07 smithi087 ceph-mon[32385]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:07.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.767 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:07 smithi100 ceph-mon[36306]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:07.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:07 smithi105 ceph-mon[35993]: pgmap v322: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:10.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:09 smithi087 ceph-mon[32385]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:10.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:09 smithi105 ceph-mon[35993]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:10.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:09 smithi100 ceph-mon[36306]: pgmap v323: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 10 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:12.083 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:11 smithi087 ceph-mon[32385]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:12.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:11 smithi105 ceph-mon[35993]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:12.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:11 smithi100 ceph-mon[36306]: pgmap v324: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 2 B/s, 0 objects/s recovering 2024-03-21T08:56:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 -- 172.21.15.87:0/1868348073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102470 msgr2=0x7f7674102850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 --2- 172.21.15.87:0/1868348073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102470 0x7f7674102850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f76700099c0 tx=0x7f7670009cd0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.301 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 -- 172.21.15.87:0/1868348073 shutdown_connections 2024-03-21T08:56:12.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 --2- 172.21.15.87:0/1868348073 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674107520 0x7f767410f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 --2- 172.21.15.87:0/1868348073 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102470 0x7f7674102850 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 --2- 172.21.15.87:0/1868348073 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7674102e20 0x7f7674106df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 -- 172.21.15.87:0/1868348073 >> 172.21.15.87:0/1868348073 conn(0x7f76740792b0 msgr2=0x7f76740796c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:12.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 -- 172.21.15.87:0/1868348073 shutdown_connections 2024-03-21T08:56:12.302 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.305+0000 7f767bcab700 1 -- 172.21.15.87:0/1868348073 wait complete. 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 Processor -- start 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 -- start start 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674102470 0x7f767419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 0x7f767419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7674107520 0x7f7674198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76741124b0 con 0x7f7674102e20 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7674112330 con 0x7f7674102470 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767bcab700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7674112630 con 0x7f7674107520 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f7679246700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 0x7f767419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.304 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f7679a47700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674102470 0x7f767419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f7679246700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 0x7f767419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59830/0 (socket says 172.21.15.87:59830) 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f767a248700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7674107520 0x7f7674198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.306+0000 7f7679246700 1 -- 172.21.15.87:0/2872376696 learned_addr learned my addr 172.21.15.87:0/2872376696 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679246700 1 -- 172.21.15.87:0/2872376696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7674107520 msgr2=0x7f7674198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679246700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7674107520 0x7f7674198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679246700 1 -- 172.21.15.87:0/2872376696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674102470 msgr2=0x7f767419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679246700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674102470 0x7f767419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679246700 1 -- 172.21.15.87:0/2872376696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7674199050 con 0x7f7674102e20 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679a47700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674102470 0x7f767419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f7679246700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 0x7f767419eb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f766400d950 tx=0x7f766400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664016070 con 0x7f7674102e20 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7674199340 con 0x7f7674102e20 2024-03-21T08:56:12.305 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7664004d10 con 0x7f7674102e20 2024-03-21T08:56:12.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76640056a0 con 0x7f7674102e20 2024-03-21T08:56:12.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.307+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76741a55d0 con 0x7f7674102e20 2024-03-21T08:56:12.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.308+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767410a6f0 con 0x7f7674102e20 2024-03-21T08:56:12.306 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.309+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7664004220 con 0x7f7674102e20 2024-03-21T08:56:12.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.309+0000 7f766affd700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f76600a3040 0x7f76600a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.310+0000 7f7679a47700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f76600a3040 0x7f76600a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.310+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f76640cf260 con 0x7f7674102e20 2024-03-21T08:56:12.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.310+0000 7f7679a47700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f76600a3040 0x7f76600a5500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7670000c00 tx=0x7f7670005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:12.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.314+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f766401f020 con 0x7f7674102e20 2024-03-21T08:56:12.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.517+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f7674199570 con 0x7f7674102e20 2024-03-21T08:56:12.513 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.517+0000 7f766affd700 1 -- 172.21.15.87:0/2872376696 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f7664022350 con 0x7f7674102e20 2024-03-21T08:56:12.514 INFO:teuthology.orchestra.run.smithi087.stdout:292057776165 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f76600a3040 msgr2=0x7f76600a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f76600a3040 0x7f76600a5500 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f7670000c00 tx=0x7f7670005fb0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 msgr2=0x7f767419eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 0x7f767419eb90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f766400d950 tx=0x7f766400dc60 comp rx=0 tx=0).stop 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 shutdown_connections 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f76600a3040 0x7f76600a5500 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7674102470 0x7f767419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.516 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7674107520 0x7f7674198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 --2- 172.21.15.87:0/2872376696 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7674102e20 0x7f767419eb90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.519+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 >> 172.21.15.87:0/2872376696 conn(0x7f76740792b0 msgr2=0x7f76741019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:12.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.520+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 shutdown_connections 2024-03-21T08:56:12.517 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.520+0000 7f767bcab700 1 -- 172.21.15.87:0/2872376696 wait complete. 2024-03-21T08:56:12.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1179685935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18106370 msgr2=0x7f8e1810df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1179685935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18106370 0x7f8e1810df40 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8e1000b0d0 tx=0x7f8e1000b3e0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1179685935 shutdown_connections 2024-03-21T08:56:12.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1179685935 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 0x7f8e18101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1179685935 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18106370 0x7f8e1810df40 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1179685935 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e18101cd0 0x7f8e18105c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1179685935 >> 172.21.15.87:0/1179685935 conn(0x7f8e180770e0 msgr2=0x7f8e180774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:12.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1179685935 shutdown_connections 2024-03-21T08:56:12.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.554+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1179685935 wait complete. 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 Processor -- start 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 -- start start 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 0x7f8e18102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 0x7f8e18103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e18106370 0x7f8e181037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e18111360 con 0x7f8e18101cd0 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181111e0 con 0x7f8e18101320 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1eeff700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e181114e0 con 0x7f8e18106370 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 0x7f8e18103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 0x7f8e18103270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59848/0 (socket says 172.21.15.87:59848) 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1cc9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 0x7f8e18102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 -- 172.21.15.87:0/1584430755 learned_addr learned my addr 172.21.15.87:0/1584430755 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:12.553 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1d49c700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e18106370 0x7f8e181037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 -- 172.21.15.87:0/1584430755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e18106370 msgr2=0x7f8e181037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e18106370 0x7f8e181037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 -- 172.21.15.87:0/1584430755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 msgr2=0x7f8e18102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 0x7f8e18102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e17fff700 1 -- 172.21.15.87:0/1584430755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e181a6e00 con 0x7f8e18101cd0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.555+0000 7f8e1cc9b700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 0x7f8e18102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.556+0000 7f8e17fff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 0x7f8e18103270 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400e970 tx=0x7f8e0400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.556+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04004a30 con 0x7f8e18101cd0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.556+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8e0400ccc0 con 0x7f8e18101cd0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.556+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e181a70f0 con 0x7f8e18101cd0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.556+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e04005650 con 0x7f8e18101cd0 2024-03-21T08:56:12.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.556+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e181a7680 con 0x7f8e18101cd0 2024-03-21T08:56:12.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.557+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e18068e50 con 0x7f8e18101cd0 2024-03-21T08:56:12.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.558+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f8e04004bd0 con 0x7f8e18101cd0 2024-03-21T08:56:12.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.558+0000 7f8e15ffb700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e080a30c0 0x7f8e080a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:12.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.559+0000 7f8e1cc9b700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e080a30c0 0x7f8e080a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:12.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.559+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8e040cf640 con 0x7f8e18101cd0 2024-03-21T08:56:12.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.559+0000 7f8e1cc9b700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e080a30c0 0x7f8e080a5580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c011490 tx=0x7f8e0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:12.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.563+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f8e0409c340 con 0x7f8e18101cd0 2024-03-21T08:56:12.759 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.763+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f8e1802da30 con 0x7f8e18101cd0 2024-03-21T08:56:12.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.764+0000 7f8e15ffb700 1 -- 172.21.15.87:0/1584430755 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f8e0409bc40 con 0x7f8e18101cd0 2024-03-21T08:56:12.760 INFO:teuthology.orchestra.run.smithi087.stdout:141733920830 2024-03-21T08:56:12.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e080a30c0 msgr2=0x7f8e080a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e080a30c0 0x7f8e080a5580 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c011490 tx=0x7f8e0c009260 comp rx=0 tx=0).stop 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 msgr2=0x7f8e18103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 0x7f8e18103270 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400e970 tx=0x7f8e0400ec80 comp rx=0 tx=0).stop 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 shutdown_connections 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8e080a30c0 0x7f8e080a5580 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8e18101320 0x7f8e18102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8e18106370 0x7f8e181037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 --2- 172.21.15.87:0/1584430755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8e18101cd0 0x7f8e18103270 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 >> 172.21.15.87:0/1584430755 conn(0x7f8e180770e0 msgr2=0x7f8e1806a910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 shutdown_connections 2024-03-21T08:56:12.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:12.766+0000 7f8e1eeff700 1 -- 172.21.15.87:0/1584430755 wait complete. 2024-03-21T08:56:13.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:12 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2872376696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T08:56:13.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:12 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1584430755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T08:56:13.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:12 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2872376696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T08:56:13.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:12 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1584430755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T08:56:13.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:12 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2872376696' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-03-21T08:56:13.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:12 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1584430755' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-21T08:56:13.617 INFO:tasks.cephadm.ceph_manager.ceph:need seq 292057776160 got 292057776165 for osd.9 2024-03-21T08:56:13.617 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:14.183 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:13 smithi100 ceph-mon[36306]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s; 1 B/s, 0 objects/s recovering 2024-03-21T08:56:14.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:13 smithi105 ceph-mon[35993]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s; 1 B/s, 0 objects/s recovering 2024-03-21T08:56:14.267 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:13 smithi087 ceph-mon[32385]: pgmap v325: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 10 op/s; 1 B/s, 0 objects/s recovering 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 -- 172.21.15.87:0/2807932464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4078101350 msgr2=0x7f4078101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 --2- 172.21.15.87:0/2807932464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4078101350 0x7f4078101730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f40680099c0 tx=0x7f4068009cd0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 -- 172.21.15.87:0/2807932464 shutdown_connections 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 --2- 172.21.15.87:0/2807932464 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40781063a0 0x7f407810df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 --2- 172.21.15.87:0/2807932464 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4078101350 0x7f4078101730 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 --2- 172.21.15.87:0/2807932464 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101d00 0x7f4078105cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 -- 172.21.15.87:0/2807932464 >> 172.21.15.87:0/2807932464 conn(0x7f40780770e0 msgr2=0x7f40780774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 -- 172.21.15.87:0/2807932464 shutdown_connections 2024-03-21T08:56:14.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 -- 172.21.15.87:0/2807932464 wait complete. 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.273+0000 7f407dde3700 1 Processor -- start 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 -- start start 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 0x7f407819e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4078101d00 0x7f407819eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40781063a0 0x7f40781987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40781113d0 con 0x7f40781063a0 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078111250 con 0x7f4078101d00 2024-03-21T08:56:14.271 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f407dde3700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4078111550 con 0x7f4078101350 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f4077fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40781063a0 0x7f40781987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 0x7f407819e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 0x7f407819e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:59748/0 (socket says 172.21.15.87:59748) 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f4076ffd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4078101d00 0x7f407819eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f4077fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40781063a0 0x7f40781987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59876/0 (socket says 172.21.15.87:59876) 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 -- 172.21.15.87:0/4071194291 learned_addr learned my addr 172.21.15.87:0/4071194291 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 -- 172.21.15.87:0/4071194291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4078101d00 msgr2=0x7f407819eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4078101d00 0x7f407819eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 -- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40781063a0 msgr2=0x7f40781987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40781063a0 0x7f40781987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 -- 172.21.15.87:0/4071194291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4078198fb0 con 0x7f4078101350 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.274+0000 7f40777fe700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 0x7f407819e600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4068006010 tx=0x7f4068003a90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.275+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4068003e50 con 0x7f4078101350 2024-03-21T08:56:14.272 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.275+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4078199240 con 0x7f4078101350 2024-03-21T08:56:14.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.275+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40680055a0 con 0x7f4078101350 2024-03-21T08:56:14.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.275+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f406800fdd0 con 0x7f4078101350 2024-03-21T08:56:14.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.275+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40781a5460 con 0x7f4078101350 2024-03-21T08:56:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.276+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407806a0a0 con 0x7f4078101350 2024-03-21T08:56:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.277+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f406800f4f0 con 0x7f4078101350 2024-03-21T08:56:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.277+0000 7f4074ff9700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40600a31e0 0x7f40600a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.277+0000 7f4076ffd700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40600a31e0 0x7f40600a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.278+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f40680d0580 con 0x7f4078101350 2024-03-21T08:56:14.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.278+0000 7f4076ffd700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40600a31e0 0x7f40600a56a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f407806b950 tx=0x7f406c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:14.280 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.285+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4068013070 con 0x7f4078101350 2024-03-21T08:56:14.480 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.484+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f40781027a0 con 0x7f4078101350 2024-03-21T08:56:14.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.485+0000 7f4074ff9700 1 -- 172.21.15.87:0/4071194291 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f406809d2b0 con 0x7f4078101350 2024-03-21T08:56:14.481 INFO:teuthology.orchestra.run.smithi087.stdout:201863462965 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40600a31e0 msgr2=0x7f40600a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40600a31e0 0x7f40600a56a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f407806b950 tx=0x7f406c009260 comp rx=0 tx=0).stop 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 msgr2=0x7f407819e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 0x7f407819e600 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4068006010 tx=0x7f4068003a90 comp rx=0 tx=0).stop 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 shutdown_connections 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40600a31e0 0x7f40600a56a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.483 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40781063a0 0x7f40781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4078101350 0x7f407819e600 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 --2- 172.21.15.87:0/4071194291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4078101d00 0x7f407819eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 >> 172.21.15.87:0/4071194291 conn(0x7f40780770e0 msgr2=0x7f4078104fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 shutdown_connections 2024-03-21T08:56:14.484 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.487+0000 7f407dde3700 1 -- 172.21.15.87:0/4071194291 wait complete. 2024-03-21T08:56:14.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 -- 172.21.15.87:0/4081380963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c102d80 msgr2=0x7f344c106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 --2- 172.21.15.87:0/4081380963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c102d80 0x7f344c106dd0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3440009920 tx=0x7f3440009c30 comp rx=0 tx=0).stop 2024-03-21T08:56:14.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 -- 172.21.15.87:0/4081380963 shutdown_connections 2024-03-21T08:56:14.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 --2- 172.21.15.87:0/4081380963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c1023d0 0x7f344c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 --2- 172.21.15.87:0/4081380963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1074b0 0x7f344c10f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 --2- 172.21.15.87:0/4081380963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c102d80 0x7f344c106dd0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 -- 172.21.15.87:0/4081380963 >> 172.21.15.87:0/4081380963 conn(0x7f344c0fdc40 msgr2=0x7f344c100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:14.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 -- 172.21.15.87:0/4081380963 shutdown_connections 2024-03-21T08:56:14.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.500+0000 7f3450fb4700 1 -- 172.21.15.87:0/4081380963 wait complete. 2024-03-21T08:56:14.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 Processor -- start 2024-03-21T08:56:14.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 -- start start 2024-03-21T08:56:14.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 0x7f344c19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c102d80 0x7f344c19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c1074b0 0x7f344c198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c1124c0 con 0x7f344c1023d0 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c112340 con 0x7f344c102d80 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.501+0000 7f3450fb4700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f344c112640 con 0x7f344c1074b0 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 0x7f344c19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 0x7f344c19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59894/0 (socket says 172.21.15.87:59894) 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 -- 172.21.15.87:0/2607496457 learned_addr learned my addr 172.21.15.87:0/2607496457 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f3449d9b700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c102d80 0x7f344c19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344ad9d700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c1074b0 0x7f344c198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 -- 172.21.15.87:0/2607496457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c1074b0 msgr2=0x7f344c198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c1074b0 0x7f344c198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 -- 172.21.15.87:0/2607496457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c102d80 msgr2=0x7f344c19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c102d80 0x7f344c19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 -- 172.21.15.87:0/2607496457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f344c199060 con 0x7f344c1023d0 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f344a59c700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 0x7f344c19e6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f343c00e9a0 tx=0x7f343c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c00ca10 con 0x7f344c1023d0 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f344c199350 con 0x7f344c1023d0 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c1a5510 con 0x7f344c1023d0 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f343c00cbb0 con 0x7f344c1023d0 2024-03-21T08:56:14.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.502+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f343c018680 con 0x7f344c1023d0 2024-03-21T08:56:14.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.503+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c1038b0 con 0x7f344c1023d0 2024-03-21T08:56:14.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.505+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f343c010430 con 0x7f344c1023d0 2024-03-21T08:56:14.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.505+0000 7f343b7fe700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f34340a3040 0x7f34340a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:14.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.505+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f343c0cfc20 con 0x7f344c1023d0 2024-03-21T08:56:14.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.505+0000 7f3449d9b700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f34340a3040 0x7f34340a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:14.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.506+0000 7f3449d9b700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f34340a3040 0x7f34340a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f344000c010 tx=0x7f3440019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:14.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.513+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f343c09c920 con 0x7f344c1023d0 2024-03-21T08:56:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.715+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f344c199d60 con 0x7f344c1023d0 2024-03-21T08:56:14.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.715+0000 7f343b7fe700 1 -- 172.21.15.87:0/2607496457 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f343c09c220 con 0x7f344c1023d0 2024-03-21T08:56:14.712 INFO:teuthology.orchestra.run.smithi087.stdout:223338299440 2024-03-21T08:56:14.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f34340a3040 msgr2=0x7f34340a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f34340a3040 0x7f34340a5500 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f344000c010 tx=0x7f3440019040 comp rx=0 tx=0).stop 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 msgr2=0x7f344c19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 0x7f344c19e6b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f343c00e9a0 tx=0x7f343c00ecb0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 shutdown_connections 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f34340a3040 0x7f34340a5500 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f344c1074b0 0x7f344c198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f344c1023d0 0x7f344c19e6b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 --2- 172.21.15.87:0/2607496457 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f344c102d80 0x7f344c19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 >> 172.21.15.87:0/2607496457 conn(0x7f344c0fdc40 msgr2=0x7f344c0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 shutdown_connections 2024-03-21T08:56:14.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:14.717+0000 7f3450fb4700 1 -- 172.21.15.87:0/2607496457 wait complete. 2024-03-21T08:56:15.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:14 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4071194291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T08:56:15.042 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:14 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2607496457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T08:56:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:14 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4071194291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T08:56:15.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:14 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2607496457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T08:56:15.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:14 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4071194291' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-21T08:56:15.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:14 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2607496457' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-21T08:56:15.420 INFO:tasks.cephadm.ceph_manager.ceph:need seq 223338299435 got 223338299440 for osd.7 2024-03-21T08:56:15.420 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:16.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:15 smithi105 ceph-mon[35993]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-21T08:56:16.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:15 smithi100 ceph-mon[36306]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-21T08:56:16.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:15 smithi087 ceph-mon[32385]: pgmap v326: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 objects/s recovering 2024-03-21T08:56:16.352 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920830 for osd.4 2024-03-21T08:56:16.352 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:17.208 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462960 got 201863462965 for osd.6 2024-03-21T08:56:17.208 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 -- 172.21.15.87:0/654532128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 msgr2=0x7f3b7c105c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/654532128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c105c90 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f3b700099c0 tx=0x7f3b70009cd0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 -- 172.21.15.87:0/654532128 shutdown_connections 2024-03-21T08:56:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/654532128 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 0x7f3b7c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/654532128 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c101700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/654532128 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c105c90 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 -- 172.21.15.87:0/654532128 >> 172.21.15.87:0/654532128 conn(0x7f3b7c0770e0 msgr2=0x7f3b7c0774f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 -- 172.21.15.87:0/654532128 shutdown_connections 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.405+0000 7f3b81a0f700 1 -- 172.21.15.87:0/654532128 wait complete. 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 Processor -- start 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 -- start start 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c102d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c103270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.404 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 0x7f3b7c1037b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c111350 con 0x7f3b7c101cd0 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1111d0 con 0x7f3b7c106370 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b81a0f700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b7c1114d0 con 0x7f3b7c101320 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b7a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c103270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b7affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c102d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.406+0000 7f3b7a7fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c103270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59914/0 (socket says 172.21.15.87:59914) 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7affd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c102d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:59798/0 (socket says 172.21.15.87:59798) 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 -- 172.21.15.87:0/4194529674 learned_addr learned my addr 172.21.15.87:0/4194529674 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7b7fe700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 0x7f3b7c1037b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 -- 172.21.15.87:0/4194529674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 msgr2=0x7f3b7c102d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c102d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 -- 172.21.15.87:0/4194529674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 msgr2=0x7f3b7c1037b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 0x7f3b7c1037b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 -- 172.21.15.87:0/4194529674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b7c1a6e80 con 0x7f3b7c101cd0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7affd700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7b7fe700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 0x7f3b7c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b7a7fc700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c103270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b70006010 tx=0x7f3b70003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b70003680 con 0x7f3b7c101cd0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3b70003e50 con 0x7f3b7c101cd0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b7c1a7110 con 0x7f3b7c101cd0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b7000fe10 con 0x7f3b7c101cd0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.407+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b7c1a75c0 con 0x7f3b7c101cd0 2024-03-21T08:56:17.406 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.408+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b7c068e50 con 0x7f3b7c101cd0 2024-03-21T08:56:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.409+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f3b7000f4f0 con 0x7f3b7c101cd0 2024-03-21T08:56:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.410+0000 7f3b80a0d700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3b640a3260 0x7f3b640a5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.407 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.410+0000 7f3b7affd700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3b640a3260 0x7f3b640a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.410+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3b700d0140 con 0x7f3b7c101cd0 2024-03-21T08:56:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.410+0000 7f3b7affd700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3b640a3260 0x7f3b640a5720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c011490 tx=0x7f3b6c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:17.408 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.413+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3b70013070 con 0x7f3b7c101cd0 2024-03-21T08:56:17.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.614+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f3b7c02da30 con 0x7f3b7c101cd0 2024-03-21T08:56:17.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.614+0000 7f3b80a0d700 1 -- 172.21.15.87:0/4194529674 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3b7009ced0 con 0x7f3b7c101cd0 2024-03-21T08:56:17.609 INFO:teuthology.orchestra.run.smithi087.stdout:34359738451 2024-03-21T08:56:17.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3b640a3260 msgr2=0x7f3b640a5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3b640a3260 0x7f3b640a5720 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c011490 tx=0x7f3b6c009260 comp rx=0 tx=0).stop 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 msgr2=0x7f3b7c103270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c103270 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3b70006010 tx=0x7f3b70003a90 comp rx=0 tx=0).stop 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 shutdown_connections 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3b640a3260 0x7f3b640a5720 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3b7c106370 0x7f3b7c1037b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3b7c101320 0x7f3b7c102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 --2- 172.21.15.87:0/4194529674 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3b7c101cd0 0x7f3b7c103270 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.616+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 >> 172.21.15.87:0/4194529674 conn(0x7f3b7c0770e0 msgr2=0x7f3b7c06a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.617+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 shutdown_connections 2024-03-21T08:56:17.612 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.617+0000 7f3b81a0f700 1 -- 172.21.15.87:0/4194529674 wait complete. 2024-03-21T08:56:17.627 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 -- 172.21.15.87:0/1200051192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102bd0 msgr2=0x7f18a810f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 --2- 172.21.15.87:0/1200051192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102bd0 0x7f18a810f030 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f18a000b0a0 tx=0x7f18a000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 -- 172.21.15.87:0/1200051192 shutdown_connections 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 --2- 172.21.15.87:0/1200051192 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a81082f0 0x7f18a81086d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 --2- 172.21.15.87:0/1200051192 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102bd0 0x7f18a810f030 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 --2- 172.21.15.87:0/1200051192 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a8102210 0x7f18a8102690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 -- 172.21.15.87:0/1200051192 >> 172.21.15.87:0/1200051192 conn(0x7f18a8079250 msgr2=0x7f18a8079660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 -- 172.21.15.87:0/1200051192 shutdown_connections 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.626+0000 7f18acf1d700 1 -- 172.21.15.87:0/1200051192 wait complete. 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 Processor -- start 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 -- start start 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 0x7f18a819e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a8102bd0 0x7f18a819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.628 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81082f0 0x7f18a8198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8112450 con 0x7f18a8102210 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81122d0 con 0x7f18a8102bd0 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18acf1d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a81125d0 con 0x7f18a81082f0 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 0x7f18a819e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 0x7f18a819e5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49426/0 (socket says 172.21.15.87:49426) 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 -- 172.21.15.87:0/676194744 learned_addr learned my addr 172.21.15.87:0/676194744 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a5d9b700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a8102bd0 0x7f18a819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a6d9d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81082f0 0x7f18a8198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 -- 172.21.15.87:0/676194744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81082f0 msgr2=0x7f18a8198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81082f0 0x7f18a8198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 -- 172.21.15.87:0/676194744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a8102bd0 msgr2=0x7f18a819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a8102bd0 0x7f18a819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a659c700 1 -- 172.21.15.87:0/676194744 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8198f70 con 0x7f18a8102210 2024-03-21T08:56:17.629 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.627+0000 7f18a5d9b700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a8102bd0 0x7f18a819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.628+0000 7f18a659c700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 0x7f18a819e5c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e9a0 tx=0x7f189c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.628+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c00ca10 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.628+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a8199260 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.628+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f189c00cbb0 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.628+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189c018680 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.628+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a81a5540 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.629+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a81038d0 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.630+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f189c010430 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.631+0000 7f18977fe700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f18900a3040 0x7f18900a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.631+0000 7f18a5d9b700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f18900a3040 0x7f18900a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.631+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f189c0cf6f0 con 0x7f18a8102210 2024-03-21T08:56:17.630 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.631+0000 7f18a5d9b700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f18900a3040 0x7f18900a5500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f18a8199e50 tx=0x7f1898009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:17.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.637+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f189c09c3f0 con 0x7f18a8102210 2024-03-21T08:56:17.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.841+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7f18a8199bc0 con 0x7f18a8102210 2024-03-21T08:56:17.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.842+0000 7f18977fe700 1 -- 172.21.15.87:0/676194744 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f189c09bcf0 con 0x7f18a8102210 2024-03-21T08:56:17.837 INFO:teuthology.orchestra.run.smithi087.stdout:257698037802 2024-03-21T08:56:17.839 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f18900a3040 msgr2=0x7f18900a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f18900a3040 0x7f18900a5500 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f18a8199e50 tx=0x7f1898009260 comp rx=0 tx=0).stop 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 msgr2=0x7f18a819e5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 0x7f18a819e5c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f189c00e9a0 tx=0x7f189c00ecb0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 shutdown_connections 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f18900a3040 0x7f18900a5500 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f18a81082f0 0x7f18a8198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f18a8102210 0x7f18a819e5c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 --2- 172.21.15.87:0/676194744 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f18a8102bd0 0x7f18a819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 >> 172.21.15.87:0/676194744 conn(0x7f18a8079250 msgr2=0x7f18a8100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 shutdown_connections 2024-03-21T08:56:17.840 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:17.844+0000 7f18acf1d700 1 -- 172.21.15.87:0/676194744 wait complete. 2024-03-21T08:56:18.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:17 smithi087 ceph-mon[32385]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:18.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:17 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4194529674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T08:56:18.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:17 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/676194744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T08:56:18.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:17 smithi105 ceph-mon[35993]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:18.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:17 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4194529674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T08:56:18.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:17 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/676194744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T08:56:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:17 smithi100 ceph-mon[36306]: pgmap v327: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:17 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4194529674' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-21T08:56:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:17 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/676194744' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-03-21T08:56:18.514 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738451 for osd.0 2024-03-21T08:56:18.515 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:18.958 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037798 got 257698037802 for osd.8 2024-03-21T08:56:18.958 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:19.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 -- 172.21.15.87:0/2589334108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54107200 msgr2=0x7f7d541075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/2589334108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54107200 0x7f7d541075e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00b0a0 tx=0x7f7d4c00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 -- 172.21.15.87:0/2589334108 shutdown_connections 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/2589334108 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d54101140 0x7f7d5410f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/2589334108 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54107200 0x7f7d541075e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/2589334108 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d54100780 0x7f7d54100c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 -- 172.21.15.87:0/2589334108 >> 172.21.15.87:0/2589334108 conn(0x7f7d54077690 msgr2=0x7f7d540feab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 -- 172.21.15.87:0/2589334108 shutdown_connections 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 -- 172.21.15.87:0/2589334108 wait complete. 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.759+0000 7f7d59a8d700 1 Processor -- start 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 -- start start 2024-03-21T08:56:19.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d54100780 0x7f7d5419e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 0x7f7d5419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d54107200 0x7f7d54198860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d541125e0 con 0x7f7d54101140 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54112460 con 0x7f7d54107200 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d59a8d700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d54112760 con 0x7f7d54100780 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 0x7f7d5419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d537fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d54100780 0x7f7d5419e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 0x7f7d5419ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49446/0 (socket says 172.21.15.87:49446) 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 -- 172.21.15.87:0/3557718367 learned_addr learned my addr 172.21.15.87:0/3557718367 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d53fff700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d54107200 0x7f7d54198860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 -- 172.21.15.87:0/3557718367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d54100780 msgr2=0x7f7d5419e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d54100780 0x7f7d5419e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 -- 172.21.15.87:0/3557718367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d54107200 msgr2=0x7f7d54198860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d54107200 0x7f7d54198860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 -- 172.21.15.87:0/3557718367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d54198fc0 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.760+0000 7f7d52ffd700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 0x7f7d5419ebe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00d980 tx=0x7f7d3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.761+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c017070 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.761+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d541992b0 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.761+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7d3c005020 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.761+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c009a40 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.761+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d541a5570 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.762+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5410a7d0 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.763+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f7d3c0051d0 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.763+0000 7f7d50ff9700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7d400a3040 0x7f7d400a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.763+0000 7f7d537fe700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7d400a3040 0x7f7d400a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.764+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7d3c0cf3a0 con 0x7f7d54101140 2024-03-21T08:56:19.773 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.764+0000 7f7d537fe700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7d400a3040 0x7f7d400a5500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009fd0 tx=0x7f7d4c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:19.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.768+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7d3c020020 con 0x7f7d54101140 2024-03-21T08:56:19.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.975+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7d54199d90 con 0x7f7d54101140 2024-03-21T08:56:19.970 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.975+0000 7f7d50ff9700 1 -- 172.21.15.87:0/3557718367 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7d3c09c0a0 con 0x7f7d54101140 2024-03-21T08:56:19.971 INFO:teuthology.orchestra.run.smithi087.stdout:55834574927 2024-03-21T08:56:19.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7d400a3040 msgr2=0x7f7d400a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7d400a3040 0x7f7d400a5500 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c009fd0 tx=0x7f7d4c00b7b0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 msgr2=0x7f7d5419ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 0x7f7d5419ebe0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00d980 tx=0x7f7d3c00dc90 comp rx=0 tx=0).stop 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 shutdown_connections 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7d400a3040 0x7f7d400a5500 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7d54107200 0x7f7d54198860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7d54100780 0x7f7d5419e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 --2- 172.21.15.87:0/3557718367 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7d54101140 0x7f7d5419ebe0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.977+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 >> 172.21.15.87:0/3557718367 conn(0x7f7d54077690 msgr2=0x7f7d541051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.978+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 shutdown_connections 2024-03-21T08:56:19.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:19.978+0000 7f7d59a8d700 1 -- 172.21.15.87:0/3557718367 wait complete. 2024-03-21T08:56:20.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:19 smithi105 ceph-mon[35993]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:20.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:19 smithi100 ceph-mon[36306]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:20.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:19 smithi087 ceph-mon[32385]: pgmap v328: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:20.730 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574921 got 55834574927 for osd.1 2024-03-21T08:56:20.730 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.003+0000 7f5e400c9700 1 -- 172.21.15.87:0/2836578720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e3806b2d0 msgr2=0x7f5e3810f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.003+0000 7f5e400c9700 1 --2- 172.21.15.87:0/2836578720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e3806b2d0 0x7f5e3810f020 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b0a0 tx=0x7f5e3000b3b0 comp rx=0 tx=0).stop 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.003+0000 7f5e400c9700 1 -- 172.21.15.87:0/2836578720 shutdown_connections 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.003+0000 7f5e400c9700 1 --2- 172.21.15.87:0/2836578720 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e381071a0 0x7f5e38107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.003+0000 7f5e400c9700 1 --2- 172.21.15.87:0/2836578720 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e3806b2d0 0x7f5e3810f020 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.003+0000 7f5e400c9700 1 --2- 172.21.15.87:0/2836578720 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e3806a910 0x7f5e3806ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.004+0000 7f5e400c9700 1 -- 172.21.15.87:0/2836578720 >> 172.21.15.87:0/2836578720 conn(0x7f5e380780f0 msgr2=0x7f5e38078500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.004+0000 7f5e400c9700 1 -- 172.21.15.87:0/2836578720 shutdown_connections 2024-03-21T08:56:20.999 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.004+0000 7f5e400c9700 1 -- 172.21.15.87:0/2836578720 wait complete. 2024-03-21T08:56:21.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.004+0000 7f5e400c9700 1 Processor -- start 2024-03-21T08:56:21.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.004+0000 7f5e400c9700 1 -- start start 2024-03-21T08:56:21.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3806a910 0x7f5e3819e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:21.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e3806b2d0 0x7f5e3819eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:21.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 0x7f5e381987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e381125c0 con 0x7f5e381071a0 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38112440 con 0x7f5e3806a910 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38112740 con 0x7f5e3806b2d0 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 0x7f5e381987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3de65700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3806a910 0x7f5e3819e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 0x7f5e381987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49456/0 (socket says 172.21.15.87:49456) 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 -- 172.21.15.87:0/3212866333 learned_addr learned my addr 172.21.15.87:0/3212866333 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3d664700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e3806b2d0 0x7f5e3819eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 -- 172.21.15.87:0/3212866333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e3806b2d0 msgr2=0x7f5e3819eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e3806b2d0 0x7f5e3819eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 -- 172.21.15.87:0/3212866333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3806a910 msgr2=0x7f5e3819e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3806a910 0x7f5e3819e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:21.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 -- 172.21.15.87:0/3212866333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38198fa0 con 0x7f5e381071a0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e3e666700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 0x7f5e381987c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b780 tx=0x7f5e30004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30021070 con 0x7f5e381071a0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e30007b80 con 0x7f5e381071a0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e38199230 con 0x7f5e381071a0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e300052a0 con 0x7f5e381071a0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.005+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e381a54a0 con 0x7f5e381071a0 2024-03-21T08:56:21.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.006+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e38103900 con 0x7f5e381071a0 2024-03-21T08:56:21.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.008+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f5e30024070 con 0x7f5e381071a0 2024-03-21T08:56:21.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.008+0000 7f5e2effd700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e240a30c0 0x7f5e240a5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:21.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.008+0000 7f5e3de65700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e240a30c0 0x7f5e240a5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:21.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.008+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5e300d04f0 con 0x7f5e381071a0 2024-03-21T08:56:21.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.009+0000 7f5e3de65700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e240a30c0 0x7f5e240a5580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009c60 tx=0x7f5e28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:21.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.012+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e30007d30 con 0x7f5e381071a0 2024-03-21T08:56:21.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:20 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3557718367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T08:56:21.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.215+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7f5e38068e50 con 0x7f5e381071a0 2024-03-21T08:56:21.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.216+0000 7f5e2effd700 1 -- 172.21.15.87:0/3212866333 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f5e3009d280 con 0x7f5e381071a0 2024-03-21T08:56:21.211 INFO:teuthology.orchestra.run.smithi087.stdout:313532612641 2024-03-21T08:56:21.214 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e240a30c0 msgr2=0x7f5e240a5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:21.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e240a30c0 0x7f5e240a5580 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009c60 tx=0x7f5e28009260 comp rx=0 tx=0).stop 2024-03-21T08:56:21.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 msgr2=0x7f5e381987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:21.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 0x7f5e381987c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b780 tx=0x7f5e30004d00 comp rx=0 tx=0).stop 2024-03-21T08:56:21.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 shutdown_connections 2024-03-21T08:56:21.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e240a30c0 0x7f5e240a5580 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:21.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e3806a910 0x7f5e3819e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:21.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e381071a0 0x7f5e381987c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:21.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 --2- 172.21.15.87:0/3212866333 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e3806b2d0 0x7f5e3819eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:21.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 >> 172.21.15.87:0/3212866333 conn(0x7f5e380780f0 msgr2=0x7f5e380ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:21.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 shutdown_connections 2024-03-21T08:56:21.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:21.218+0000 7f5e400c9700 1 -- 172.21.15.87:0/3212866333 wait complete. 2024-03-21T08:56:21.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:20 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3557718367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T08:56:21.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:20 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3557718367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-21T08:56:22.034 INFO:tasks.cephadm.ceph_manager.ceph:need seq 313532612632 got 313532612641 for osd.10 2024-03-21T08:56:22.035 DEBUG:teuthology.parallel:result is None 2024-03-21T08:56:22.035 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-21T08:56:22.035 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T08:56:22.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:21 smithi105 ceph-mon[35993]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:22.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:21 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3212866333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T08:56:22.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:21 smithi100 ceph-mon[36306]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:22.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:21 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3212866333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T08:56:22.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:21 smithi087 ceph-mon[32385]: pgmap v329: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:22.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:21 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3212866333' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-03-21T08:56:22.418 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 -- 172.21.15.87:0/1799212795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 msgr2=0x7f31381074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 --2- 172.21.15.87:0/1799212795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381074b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3134009990 tx=0x7f3134009ca0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.124 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 -- 172.21.15.87:0/1799212795 shutdown_connections 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 --2- 172.21.15.87:0/1799212795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3138101a30 0x7f313810df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 --2- 172.21.15.87:0/1799212795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381074b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 --2- 172.21.15.87:0/1799212795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138101090 0x7f31381014f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 -- 172.21.15.87:0/1799212795 >> 172.21.15.87:0/1799212795 conn(0x7f3138077fc0 msgr2=0x7f31380783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 -- 172.21.15.87:0/1799212795 shutdown_connections 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.128+0000 7f314023c700 1 -- 172.21.15.87:0/1799212795 wait complete. 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 Processor -- start 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 -- start start 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3138101090 0x7f313819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:24.125 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138101a30 0x7f313819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138111380 con 0x7f31381070d0 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138111200 con 0x7f3138101a30 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f314023c700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3138111500 con 0x7f3138101090 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f313e7d9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f313e7d9700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381987c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49474/0 (socket says 172.21.15.87:49474) 2024-03-21T08:56:24.126 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.129+0000 7f313e7d9700 1 -- 172.21.15.87:0/182378827 learned_addr learned my addr 172.21.15.87:0/182378827 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313dfd8700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3138101090 0x7f313819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313e7d9700 1 -- 172.21.15.87:0/182378827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3138101090 msgr2=0x7f313819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313e7d9700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3138101090 0x7f313819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.127 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313e7d9700 1 -- 172.21.15.87:0/182378827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138101a30 msgr2=0x7f313819ebe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313e7d9700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138101a30 0x7f313819ebe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313e7d9700 1 -- 172.21.15.87:0/182378827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31381990a0 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f313e7d9700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381987c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f313000ef30 tx=0x7f313000c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300092e0 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3130004bb0 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3138199330 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3130004280 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.130+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31381a5450 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.131+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3138102780 con 0x7f31381070d0 2024-03-21T08:56:24.128 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.133+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7f313001b030 con 0x7f31381070d0 2024-03-21T08:56:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.133+0000 7f312effd700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31240a3040 0x7f31240a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.133+0000 7f313dfd8700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31240a3040 0x7f31240a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.133+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31300cfad0 con 0x7f31381070d0 2024-03-21T08:56:24.129 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.134+0000 7f313dfd8700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31240a3040 0x7f31240a5500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f313400b2e0 tx=0x7f3134000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:24.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.137+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3130022020 con 0x7f31381070d0 2024-03-21T08:56:24.132 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:23 smithi087 ceph-mon[32385]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:24.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:23 smithi105 ceph-mon[35993]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:24.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:23 smithi100 ceph-mon[36306]: pgmap v330: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:24.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.358+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3138104e80 con 0x7f31240a3040 2024-03-21T08:56:24.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.366+0000 7f312effd700 1 -- 172.21.15.87:0/182378827 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258981 (secure 0 0 0) 0x7f3138104e80 con 0x7f31240a3040 2024-03-21T08:56:24.366 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:24.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31240a3040 msgr2=0x7f31240a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:24.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31240a3040 0x7f31240a5500 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f313400b2e0 tx=0x7f3134000bc0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 msgr2=0x7f31381987c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381987c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f313000ef30 tx=0x7f313000c450 comp rx=0 tx=0).stop 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 shutdown_connections 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31240a3040 0x7f31240a5500 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f31381070d0 0x7f31381987c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3138101090 0x7f313819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 --2- 172.21.15.87:0/182378827 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3138101a30 0x7f313819ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 >> 172.21.15.87:0/182378827 conn(0x7f3138077fc0 msgr2=0x7f3138100440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 shutdown_connections 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:24.370+0000 7f314023c700 1 -- 172.21.15.87:0/182378827 wait complete. 2024-03-21T08:56:24.369 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-03-21T08:56:25.080 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":330,"stamp":"2024-03-21T08:56:22.872374+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658461,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":858,"num_read_kb":546,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356920,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584904,"statfs":{"total":1151940427776,"available":1151574941696,"internally_reserved":0,"allocated":12128256,"data_stored":7495551,"data_compressed":112533,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":125,"num_read_kb":55,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003246"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072875+0000","last_change":"2024-03-21T08:55:07.542089+0000","last_active":"2024-03-21T08:56:00.072875+0000","last_peered":"2024-03-21T08:56:00.072875+0000","last_clean":"2024-03-21T08:56:00.072875+0000","last_became_active":"2024-03-21T08:55:07.541790+0000","last_became_peered":"2024-03-21T08:55:07.541790+0000","last_unstale":"2024-03-21T08:56:00.072875+0000","last_undegraded":"2024-03-21T08:56:00.072875+0000","last_fullsized":"2024-03-21T08:56:00.072875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:57:10.863345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068003+0000","last_change":"2024-03-21T08:55:07.543892+0000","last_active":"2024-03-21T08:56:00.068003+0000","last_peered":"2024-03-21T08:56:00.068003+0000","last_clean":"2024-03-21T08:56:00.068003+0000","last_became_active":"2024-03-21T08:55:07.541230+0000","last_became_peered":"2024-03-21T08:55:07.541230+0000","last_unstale":"2024-03-21T08:56:00.068003+0000","last_undegraded":"2024-03-21T08:56:00.068003+0000","last_fullsized":"2024-03-21T08:56:00.068003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:52:35.545762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068237+0000","last_change":"2024-03-21T08:55:07.537095+0000","last_active":"2024-03-21T08:56:00.068237+0000","last_peered":"2024-03-21T08:56:00.068237+0000","last_clean":"2024-03-21T08:56:00.068237+0000","last_became_active":"2024-03-21T08:55:07.536784+0000","last_became_peered":"2024-03-21T08:55:07.536784+0000","last_unstale":"2024-03-21T08:56:00.068237+0000","last_undegraded":"2024-03-21T08:56:00.068237+0000","last_fullsized":"2024-03-21T08:56:00.068237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:42:09.373813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074809+0000","last_change":"2024-03-21T08:55:07.533330+0000","last_active":"2024-03-21T08:56:00.074809+0000","last_peered":"2024-03-21T08:56:00.074809+0000","last_clean":"2024-03-21T08:56:00.074809+0000","last_became_active":"2024-03-21T08:55:07.532840+0000","last_became_peered":"2024-03-21T08:55:07.532840+0000","last_unstale":"2024-03-21T08:56:00.074809+0000","last_undegraded":"2024-03-21T08:56:00.074809+0000","last_fullsized":"2024-03-21T08:56:00.074809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:52:36.565830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076296+0000","last_change":"2024-03-21T08:55:07.549259+0000","last_active":"2024-03-21T08:56:00.076296+0000","last_peered":"2024-03-21T08:56:00.076296+0000","last_clean":"2024-03-21T08:56:00.076296+0000","last_became_active":"2024-03-21T08:55:07.548161+0000","last_became_peered":"2024-03-21T08:55:07.548161+0000","last_unstale":"2024-03-21T08:56:00.076296+0000","last_undegraded":"2024-03-21T08:56:00.076296+0000","last_fullsized":"2024-03-21T08:56:00.076296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:12.803360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066466+0000","last_change":"2024-03-21T08:55:07.530950+0000","last_active":"2024-03-21T08:56:00.066466+0000","last_peered":"2024-03-21T08:56:00.066466+0000","last_clean":"2024-03-21T08:56:00.066466+0000","last_became_active":"2024-03-21T08:55:07.530582+0000","last_became_peered":"2024-03-21T08:55:07.530582+0000","last_unstale":"2024-03-21T08:56:00.066466+0000","last_undegraded":"2024-03-21T08:56:00.066466+0000","last_fullsized":"2024-03-21T08:56:00.066466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:12.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070232+0000","last_change":"2024-03-21T08:55:07.535591+0000","last_active":"2024-03-21T08:56:00.070232+0000","last_peered":"2024-03-21T08:56:00.070232+0000","last_clean":"2024-03-21T08:56:00.070232+0000","last_became_active":"2024-03-21T08:55:07.535257+0000","last_became_peered":"2024-03-21T08:55:07.535257+0000","last_unstale":"2024-03-21T08:56:00.070232+0000","last_undegraded":"2024-03-21T08:56:00.070232+0000","last_fullsized":"2024-03-21T08:56:00.070232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:39.383653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392635+0000","last_change":"2024-03-21T08:55:07.532165+0000","last_active":"2024-03-21T08:56:00.392635+0000","last_peered":"2024-03-21T08:56:00.392635+0000","last_clean":"2024-03-21T08:56:00.392635+0000","last_became_active":"2024-03-21T08:55:07.531822+0000","last_became_peered":"2024-03-21T08:55:07.531822+0000","last_unstale":"2024-03-21T08:56:00.392635+0000","last_undegraded":"2024-03-21T08:56:00.392635+0000","last_fullsized":"2024-03-21T08:56:00.392635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:56.245424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.077869+0000","last_change":"2024-03-21T08:56:00.077869+0000","last_active":"2024-03-21T08:56:00.077869+0000","last_peered":"2024-03-21T08:56:00.077869+0000","last_clean":"2024-03-21T08:56:00.077869+0000","last_became_active":"2024-03-21T08:56:00.077541+0000","last_became_peered":"2024-03-21T08:56:00.077541+0000","last_unstale":"2024-03-21T08:56:00.077869+0000","last_undegraded":"2024-03-21T08:56:00.077869+0000","last_fullsized":"2024-03-21T08:56:00.077869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:23.227887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066888+0000","last_change":"2024-03-21T08:55:07.532227+0000","last_active":"2024-03-21T08:56:00.066888+0000","last_peered":"2024-03-21T08:56:00.066888+0000","last_clean":"2024-03-21T08:56:00.066888+0000","last_became_active":"2024-03-21T08:55:07.531349+0000","last_became_peered":"2024-03-21T08:55:07.531349+0000","last_unstale":"2024-03-21T08:56:00.066888+0000","last_undegraded":"2024-03-21T08:56:00.066888+0000","last_fullsized":"2024-03-21T08:56:00.066888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:07:40.318544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070379+0000","last_change":"2024-03-21T08:55:07.533531+0000","last_active":"2024-03-21T08:56:00.070379+0000","last_peered":"2024-03-21T08:56:00.070379+0000","last_clean":"2024-03-21T08:56:00.070379+0000","last_became_active":"2024-03-21T08:55:07.533166+0000","last_became_peered":"2024-03-21T08:55:07.533166+0000","last_unstale":"2024-03-21T08:56:00.070379+0000","last_undegraded":"2024-03-21T08:56:00.070379+0000","last_fullsized":"2024-03-21T08:56:00.070379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:19:24.912748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074799+0000","last_change":"2024-03-21T08:55:07.533643+0000","last_active":"2024-03-21T08:56:00.074799+0000","last_peered":"2024-03-21T08:56:00.074799+0000","last_clean":"2024-03-21T08:56:00.074799+0000","last_became_active":"2024-03-21T08:55:07.533262+0000","last_became_peered":"2024-03-21T08:55:07.533262+0000","last_unstale":"2024-03-21T08:56:00.074799+0000","last_undegraded":"2024-03-21T08:56:00.074799+0000","last_fullsized":"2024-03-21T08:56:00.074799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:07.641225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069129+0000","last_change":"2024-03-21T08:55:07.524670+0000","last_active":"2024-03-21T08:56:00.069129+0000","last_peered":"2024-03-21T08:56:00.069129+0000","last_clean":"2024-03-21T08:56:00.069129+0000","last_became_active":"2024-03-21T08:55:07.524295+0000","last_became_peered":"2024-03-21T08:55:07.524295+0000","last_unstale":"2024-03-21T08:56:00.069129+0000","last_undegraded":"2024-03-21T08:56:00.069129+0000","last_fullsized":"2024-03-21T08:56:00.069129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:07.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068383+0000","last_change":"2024-03-21T08:55:07.534401+0000","last_active":"2024-03-21T08:56:00.068383+0000","last_peered":"2024-03-21T08:56:00.068383+0000","last_clean":"2024-03-21T08:56:00.068383+0000","last_became_active":"2024-03-21T08:55:07.534056+0000","last_became_peered":"2024-03-21T08:55:07.534056+0000","last_unstale":"2024-03-21T08:56:00.068383+0000","last_undegraded":"2024-03-21T08:56:00.068383+0000","last_fullsized":"2024-03-21T08:56:00.068383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:57:55.943564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.067782+0000","last_change":"2024-03-21T08:55:07.535915+0000","last_active":"2024-03-21T08:56:00.067782+0000","last_peered":"2024-03-21T08:56:00.067782+0000","last_clean":"2024-03-21T08:56:00.067782+0000","last_became_active":"2024-03-21T08:55:07.535540+0000","last_became_peered":"2024-03-21T08:55:07.535540+0000","last_unstale":"2024-03-21T08:56:00.067782+0000","last_undegraded":"2024-03-21T08:56:00.067782+0000","last_fullsized":"2024-03-21T08:56:00.067782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:30.916226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069239+0000","last_change":"2024-03-21T08:55:07.541894+0000","last_active":"2024-03-21T08:56:00.069239+0000","last_peered":"2024-03-21T08:56:00.069239+0000","last_clean":"2024-03-21T08:56:00.069239+0000","last_became_active":"2024-03-21T08:55:07.541545+0000","last_became_peered":"2024-03-21T08:55:07.541545+0000","last_unstale":"2024-03-21T08:56:00.069239+0000","last_undegraded":"2024-03-21T08:56:00.069239+0000","last_fullsized":"2024-03-21T08:56:00.069239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:11.090894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072516+0000","last_change":"2024-03-21T08:55:07.541228+0000","last_active":"2024-03-21T08:56:00.072516+0000","last_peered":"2024-03-21T08:56:00.072516+0000","last_clean":"2024-03-21T08:56:00.072516+0000","last_became_active":"2024-03-21T08:55:07.540836+0000","last_became_peered":"2024-03-21T08:55:07.540836+0000","last_unstale":"2024-03-21T08:56:00.072516+0000","last_undegraded":"2024-03-21T08:56:00.072516+0000","last_fullsized":"2024-03-21T08:56:00.072516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:34:09.491557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070181+0000","last_change":"2024-03-21T08:55:07.525778+0000","last_active":"2024-03-21T08:56:00.070181+0000","last_peered":"2024-03-21T08:56:00.070181+0000","last_clean":"2024-03-21T08:56:00.070181+0000","last_became_active":"2024-03-21T08:55:07.525469+0000","last_became_peered":"2024-03-21T08:55:07.525469+0000","last_unstale":"2024-03-21T08:56:00.070181+0000","last_undegraded":"2024-03-21T08:56:00.070181+0000","last_fullsized":"2024-03-21T08:56:00.070181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:41:31.715570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071403+0000","last_change":"2024-03-21T08:55:07.529929+0000","last_active":"2024-03-21T08:56:00.071403+0000","last_peered":"2024-03-21T08:56:00.071403+0000","last_clean":"2024-03-21T08:56:00.071403+0000","last_became_active":"2024-03-21T08:55:07.529615+0000","last_became_peered":"2024-03-21T08:55:07.529615+0000","last_unstale":"2024-03-21T08:56:00.071403+0000","last_undegraded":"2024-03-21T08:56:00.071403+0000","last_fullsized":"2024-03-21T08:56:00.071403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:42:19.543201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":100,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:18.035466+0000","last_change":"2024-03-21T08:55:06.239149+0000","last_active":"2024-03-21T08:56:18.035466+0000","last_peered":"2024-03-21T08:56:18.035466+0000","last_clean":"2024-03-21T08:56:18.035466+0000","last_became_active":"2024-03-21T08:55:06.238833+0000","last_became_peered":"2024-03-21T08:55:06.238833+0000","last_unstale":"2024-03-21T08:56:18.035466+0000","last_undegraded":"2024-03-21T08:56:18.035466+0000","last_fullsized":"2024-03-21T08:56:18.035466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:28.779442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":68,"num_read_kb":68,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393075+0000","last_change":"2024-03-21T08:55:07.533596+0000","last_active":"2024-03-21T08:56:00.393075+0000","last_peered":"2024-03-21T08:56:00.393075+0000","last_clean":"2024-03-21T08:56:00.393075+0000","last_became_active":"2024-03-21T08:55:07.533249+0000","last_became_peered":"2024-03-21T08:55:07.533249+0000","last_unstale":"2024-03-21T08:56:00.393075+0000","last_undegraded":"2024-03-21T08:56:00.393075+0000","last_fullsized":"2024-03-21T08:56:00.393075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:21:40.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393207+0000","last_change":"2024-03-21T08:55:06.227045+0000","last_active":"2024-03-21T08:56:00.393207+0000","last_peered":"2024-03-21T08:56:00.393207+0000","last_clean":"2024-03-21T08:56:00.393207+0000","last_became_active":"2024-03-21T08:55:06.226748+0000","last_became_peered":"2024-03-21T08:55:06.226748+0000","last_unstale":"2024-03-21T08:56:00.393207+0000","last_undegraded":"2024-03-21T08:56:00.393207+0000","last_fullsized":"2024-03-21T08:56:00.393207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:37.677094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068191+0000","last_change":"2024-03-21T08:55:07.540024+0000","last_active":"2024-03-21T08:56:00.068191+0000","last_peered":"2024-03-21T08:56:00.068191+0000","last_clean":"2024-03-21T08:56:00.068191+0000","last_became_active":"2024-03-21T08:55:07.537574+0000","last_became_peered":"2024-03-21T08:55:07.537574+0000","last_unstale":"2024-03-21T08:56:00.068191+0000","last_undegraded":"2024-03-21T08:56:00.068191+0000","last_fullsized":"2024-03-21T08:56:00.068191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:27:39.053045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076899+0000","last_change":"2024-03-21T08:55:06.237261+0000","last_active":"2024-03-21T08:56:00.076899+0000","last_peered":"2024-03-21T08:56:00.076899+0000","last_clean":"2024-03-21T08:56:00.076899+0000","last_became_active":"2024-03-21T08:55:06.236838+0000","last_became_peered":"2024-03-21T08:55:06.236838+0000","last_unstale":"2024-03-21T08:56:00.076899+0000","last_undegraded":"2024-03-21T08:56:00.076899+0000","last_fullsized":"2024-03-21T08:56:00.076899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:59.881075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069339+0000","last_change":"2024-03-21T08:55:07.529159+0000","last_active":"2024-03-21T08:56:00.069339+0000","last_peered":"2024-03-21T08:56:00.069339+0000","last_clean":"2024-03-21T08:56:00.069339+0000","last_became_active":"2024-03-21T08:55:07.527260+0000","last_became_peered":"2024-03-21T08:55:07.527260+0000","last_unstale":"2024-03-21T08:56:00.069339+0000","last_undegraded":"2024-03-21T08:56:00.069339+0000","last_fullsized":"2024-03-21T08:56:00.069339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:00:08.825873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076666+0000","last_change":"2024-03-21T08:55:06.232959+0000","last_active":"2024-03-21T08:56:00.076666+0000","last_peered":"2024-03-21T08:56:00.076666+0000","last_clean":"2024-03-21T08:56:00.076666+0000","last_became_active":"2024-03-21T08:55:06.232606+0000","last_became_peered":"2024-03-21T08:55:06.232606+0000","last_unstale":"2024-03-21T08:56:00.076666+0000","last_undegraded":"2024-03-21T08:56:00.076666+0000","last_fullsized":"2024-03-21T08:56:00.076666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:27:07.480767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074544+0000","last_change":"2024-03-21T08:55:07.530711+0000","last_active":"2024-03-21T08:56:00.074544+0000","last_peered":"2024-03-21T08:56:00.074544+0000","last_clean":"2024-03-21T08:56:00.074544+0000","last_became_active":"2024-03-21T08:55:07.530374+0000","last_became_peered":"2024-03-21T08:55:07.530374+0000","last_unstale":"2024-03-21T08:56:00.074544+0000","last_undegraded":"2024-03-21T08:56:00.074544+0000","last_fullsized":"2024-03-21T08:56:00.074544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:06.524172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:20.012199+0000","last_change":"2024-03-21T08:55:06.233934+0000","last_active":"2024-03-21T08:56:20.012199+0000","last_peered":"2024-03-21T08:56:20.012199+0000","last_clean":"2024-03-21T08:56:20.012199+0000","last_became_active":"2024-03-21T08:55:06.233618+0000","last_became_peered":"2024-03-21T08:55:06.233618+0000","last_unstale":"2024-03-21T08:56:20.012199+0000","last_undegraded":"2024-03-21T08:56:20.012199+0000","last_fullsized":"2024-03-21T08:56:20.012199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:08.769883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076281+0000","last_change":"2024-03-21T08:55:07.546411+0000","last_active":"2024-03-21T08:56:00.076281+0000","last_peered":"2024-03-21T08:56:00.076281+0000","last_clean":"2024-03-21T08:56:00.076281+0000","last_became_active":"2024-03-21T08:55:07.546051+0000","last_became_peered":"2024-03-21T08:55:07.546051+0000","last_unstale":"2024-03-21T08:56:00.076281+0000","last_undegraded":"2024-03-21T08:56:00.076281+0000","last_fullsized":"2024-03-21T08:56:00.076281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:04:29.400154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066447+0000","last_change":"2024-03-21T08:55:06.226791+0000","last_active":"2024-03-21T08:56:00.066447+0000","last_peered":"2024-03-21T08:56:00.066447+0000","last_clean":"2024-03-21T08:56:00.066447+0000","last_became_active":"2024-03-21T08:55:06.226438+0000","last_became_peered":"2024-03-21T08:55:06.226438+0000","last_unstale":"2024-03-21T08:56:00.066447+0000","last_undegraded":"2024-03-21T08:56:00.066447+0000","last_fullsized":"2024-03-21T08:56:00.066447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:15:03.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074475+0000","last_change":"2024-03-21T08:55:07.529623+0000","last_active":"2024-03-21T08:56:00.074475+0000","last_peered":"2024-03-21T08:56:00.074475+0000","last_clean":"2024-03-21T08:56:00.074475+0000","last_became_active":"2024-03-21T08:55:07.529293+0000","last_became_peered":"2024-03-21T08:55:07.529293+0000","last_unstale":"2024-03-21T08:56:00.074475+0000","last_undegraded":"2024-03-21T08:56:00.074475+0000","last_fullsized":"2024-03-21T08:56:00.074475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:43.265861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068728+0000","last_change":"2024-03-21T08:55:06.236381+0000","last_active":"2024-03-21T08:56:00.068728+0000","last_peered":"2024-03-21T08:56:00.068728+0000","last_clean":"2024-03-21T08:56:00.068728+0000","last_became_active":"2024-03-21T08:55:06.236071+0000","last_became_peered":"2024-03-21T08:55:06.236071+0000","last_unstale":"2024-03-21T08:56:00.068728+0000","last_undegraded":"2024-03-21T08:56:00.068728+0000","last_fullsized":"2024-03-21T08:56:00.068728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:13.338191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.073780+0000","last_change":"2024-03-21T08:56:00.073780+0000","last_active":"2024-03-21T08:56:00.073780+0000","last_peered":"2024-03-21T08:56:00.073780+0000","last_clean":"2024-03-21T08:56:00.073780+0000","last_became_active":"2024-03-21T08:56:00.073444+0000","last_became_peered":"2024-03-21T08:56:00.073444+0000","last_unstale":"2024-03-21T08:56:00.073780+0000","last_undegraded":"2024-03-21T08:56:00.073780+0000","last_fullsized":"2024-03-21T08:56:00.073780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:25.618811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074361+0000","last_change":"2024-03-21T08:55:06.233121+0000","last_active":"2024-03-21T08:56:00.074361+0000","last_peered":"2024-03-21T08:56:00.074361+0000","last_clean":"2024-03-21T08:56:00.074361+0000","last_became_active":"2024-03-21T08:55:06.232772+0000","last_became_peered":"2024-03-21T08:55:06.232772+0000","last_unstale":"2024-03-21T08:56:00.074361+0000","last_undegraded":"2024-03-21T08:56:00.074361+0000","last_fullsized":"2024-03-21T08:56:00.074361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:05.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069830+0000","last_change":"2024-03-21T08:55:07.527483+0000","last_active":"2024-03-21T08:56:00.069830+0000","last_peered":"2024-03-21T08:56:00.069830+0000","last_clean":"2024-03-21T08:56:00.069830+0000","last_became_active":"2024-03-21T08:55:07.527038+0000","last_became_peered":"2024-03-21T08:55:07.527038+0000","last_unstale":"2024-03-21T08:56:00.069830+0000","last_undegraded":"2024-03-21T08:56:00.069830+0000","last_fullsized":"2024-03-21T08:56:00.069830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:27:13.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":160,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:19.010773+0000","last_change":"2024-03-21T08:55:06.233677+0000","last_active":"2024-03-21T08:56:19.010773+0000","last_peered":"2024-03-21T08:56:19.010773+0000","last_clean":"2024-03-21T08:56:19.010773+0000","last_became_active":"2024-03-21T08:55:06.233312+0000","last_became_peered":"2024-03-21T08:55:06.233312+0000","last_unstale":"2024-03-21T08:56:19.010773+0000","last_undegraded":"2024-03-21T08:56:19.010773+0000","last_fullsized":"2024-03-21T08:56:19.010773+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:51:53.894648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":137,"num_read_kb":137,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069835+0000","last_change":"2024-03-21T08:55:07.528174+0000","last_active":"2024-03-21T08:56:00.069835+0000","last_peered":"2024-03-21T08:56:00.069835+0000","last_clean":"2024-03-21T08:56:00.069835+0000","last_became_active":"2024-03-21T08:55:07.527313+0000","last_became_peered":"2024-03-21T08:55:07.527313+0000","last_unstale":"2024-03-21T08:56:00.069835+0000","last_undegraded":"2024-03-21T08:56:00.069835+0000","last_fullsized":"2024-03-21T08:56:00.069835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:55:11.069071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070546+0000","last_change":"2024-03-21T08:55:06.237324+0000","last_active":"2024-03-21T08:56:00.070546+0000","last_peered":"2024-03-21T08:56:00.070546+0000","last_clean":"2024-03-21T08:56:00.070546+0000","last_became_active":"2024-03-21T08:55:06.235319+0000","last_became_peered":"2024-03-21T08:55:06.235319+0000","last_unstale":"2024-03-21T08:56:00.070546+0000","last_undegraded":"2024-03-21T08:56:00.070546+0000","last_fullsized":"2024-03-21T08:56:00.070546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:05:32.872612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068119+0000","last_change":"2024-03-21T08:55:07.540552+0000","last_active":"2024-03-21T08:56:00.068119+0000","last_peered":"2024-03-21T08:56:00.068119+0000","last_clean":"2024-03-21T08:56:00.068119+0000","last_became_active":"2024-03-21T08:55:07.540211+0000","last_became_peered":"2024-03-21T08:55:07.540211+0000","last_unstale":"2024-03-21T08:56:00.068119+0000","last_undegraded":"2024-03-21T08:56:00.068119+0000","last_fullsized":"2024-03-21T08:56:00.068119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:42.644523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074636+0000","last_change":"2024-03-21T08:55:06.228420+0000","last_active":"2024-03-21T08:56:00.074636+0000","last_peered":"2024-03-21T08:56:00.074636+0000","last_clean":"2024-03-21T08:56:00.074636+0000","last_became_active":"2024-03-21T08:55:06.228037+0000","last_became_peered":"2024-03-21T08:55:06.228037+0000","last_unstale":"2024-03-21T08:56:00.074636+0000","last_undegraded":"2024-03-21T08:56:00.074636+0000","last_fullsized":"2024-03-21T08:56:00.074636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:22.437321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076329+0000","last_change":"2024-03-21T08:56:00.076329+0000","last_active":"2024-03-21T08:56:00.076329+0000","last_peered":"2024-03-21T08:56:00.076329+0000","last_clean":"2024-03-21T08:56:00.076329+0000","last_became_active":"2024-03-21T08:56:00.076092+0000","last_became_peered":"2024-03-21T08:56:00.076092+0000","last_unstale":"2024-03-21T08:56:00.076329+0000","last_undegraded":"2024-03-21T08:56:00.076329+0000","last_fullsized":"2024-03-21T08:56:00.076329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:18:04.674021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":198,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:19.010875+0000","last_change":"2024-03-21T08:55:06.232681+0000","last_active":"2024-03-21T08:56:19.010875+0000","last_peered":"2024-03-21T08:56:19.010875+0000","last_clean":"2024-03-21T08:56:19.010875+0000","last_became_active":"2024-03-21T08:55:06.232109+0000","last_became_peered":"2024-03-21T08:55:06.232109+0000","last_unstale":"2024-03-21T08:56:19.010875+0000","last_undegraded":"2024-03-21T08:56:19.010875+0000","last_fullsized":"2024-03-21T08:56:19.010875+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:11.218106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":149,"num_read_kb":134,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076644+0000","last_change":"2024-03-21T08:55:07.546464+0000","last_active":"2024-03-21T08:56:00.076644+0000","last_peered":"2024-03-21T08:56:00.076644+0000","last_clean":"2024-03-21T08:56:00.076644+0000","last_became_active":"2024-03-21T08:55:07.546111+0000","last_became_peered":"2024-03-21T08:55:07.546111+0000","last_unstale":"2024-03-21T08:56:00.076644+0000","last_undegraded":"2024-03-21T08:56:00.076644+0000","last_fullsized":"2024-03-21T08:56:00.076644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:57:12.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071398+0000","last_change":"2024-03-21T08:55:06.233157+0000","last_active":"2024-03-21T08:56:00.071398+0000","last_peered":"2024-03-21T08:56:00.071398+0000","last_clean":"2024-03-21T08:56:00.071398+0000","last_became_active":"2024-03-21T08:55:06.232302+0000","last_became_peered":"2024-03-21T08:55:06.232302+0000","last_unstale":"2024-03-21T08:56:00.071398+0000","last_undegraded":"2024-03-21T08:56:00.071398+0000","last_fullsized":"2024-03-21T08:56:00.071398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:47.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068774+0000","last_change":"2024-03-21T08:55:07.531265+0000","last_active":"2024-03-21T08:56:00.068774+0000","last_peered":"2024-03-21T08:56:00.068774+0000","last_clean":"2024-03-21T08:56:00.068774+0000","last_became_active":"2024-03-21T08:55:07.530913+0000","last_became_peered":"2024-03-21T08:55:07.530913+0000","last_unstale":"2024-03-21T08:56:00.068774+0000","last_undegraded":"2024-03-21T08:56:00.068774+0000","last_fullsized":"2024-03-21T08:56:00.068774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:34.171463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074401+0000","last_change":"2024-03-21T08:55:06.233411+0000","last_active":"2024-03-21T08:56:00.074401+0000","last_peered":"2024-03-21T08:56:00.074401+0000","last_clean":"2024-03-21T08:56:00.074401+0000","last_became_active":"2024-03-21T08:55:06.233031+0000","last_became_peered":"2024-03-21T08:55:06.233031+0000","last_unstale":"2024-03-21T08:56:00.074401+0000","last_undegraded":"2024-03-21T08:56:00.074401+0000","last_fullsized":"2024-03-21T08:56:00.074401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:52:07.258184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072534+0000","last_change":"2024-03-21T08:56:00.072534+0000","last_active":"2024-03-21T08:56:00.072534+0000","last_peered":"2024-03-21T08:56:00.072534+0000","last_clean":"2024-03-21T08:56:00.072534+0000","last_became_active":"2024-03-21T08:56:00.072138+0000","last_became_peered":"2024-03-21T08:56:00.072138+0000","last_unstale":"2024-03-21T08:56:00.072534+0000","last_undegraded":"2024-03-21T08:56:00.072534+0000","last_fullsized":"2024-03-21T08:56:00.072534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:19.550293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:20.038147+0000","last_change":"2024-03-21T08:55:06.230587+0000","last_active":"2024-03-21T08:56:20.038147+0000","last_peered":"2024-03-21T08:56:20.038147+0000","last_clean":"2024-03-21T08:56:20.038147+0000","last_became_active":"2024-03-21T08:55:06.230201+0000","last_became_peered":"2024-03-21T08:55:06.230201+0000","last_unstale":"2024-03-21T08:56:20.038147+0000","last_undegraded":"2024-03-21T08:56:20.038147+0000","last_fullsized":"2024-03-21T08:56:20.038147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:01.954850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070130+0000","last_change":"2024-03-21T08:55:07.532592+0000","last_active":"2024-03-21T08:56:00.070130+0000","last_peered":"2024-03-21T08:56:00.070130+0000","last_clean":"2024-03-21T08:56:00.070130+0000","last_became_active":"2024-03-21T08:55:07.532210+0000","last_became_peered":"2024-03-21T08:55:07.532210+0000","last_unstale":"2024-03-21T08:56:00.070130+0000","last_undegraded":"2024-03-21T08:56:00.070130+0000","last_fullsized":"2024-03-21T08:56:00.070130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:59.775085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076118+0000","last_change":"2024-03-21T08:55:06.229096+0000","last_active":"2024-03-21T08:56:00.076118+0000","last_peered":"2024-03-21T08:56:00.076118+0000","last_clean":"2024-03-21T08:56:00.076118+0000","last_became_active":"2024-03-21T08:55:06.228772+0000","last_became_peered":"2024-03-21T08:55:06.228772+0000","last_unstale":"2024-03-21T08:56:00.076118+0000","last_undegraded":"2024-03-21T08:56:00.076118+0000","last_fullsized":"2024-03-21T08:56:00.076118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:05:31.017977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074681+0000","last_change":"2024-03-21T08:55:07.527968+0000","last_active":"2024-03-21T08:56:00.074681+0000","last_peered":"2024-03-21T08:56:00.074681+0000","last_clean":"2024-03-21T08:56:00.074681+0000","last_became_active":"2024-03-21T08:55:07.523326+0000","last_became_peered":"2024-03-21T08:55:07.523326+0000","last_unstale":"2024-03-21T08:56:00.074681+0000","last_undegraded":"2024-03-21T08:56:00.074681+0000","last_fullsized":"2024-03-21T08:56:00.074681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:01:22.903576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.082401+0000","last_change":"2024-03-21T08:56:00.081955+0000","last_active":"2024-03-21T08:56:00.082401+0000","last_peered":"2024-03-21T08:56:00.082401+0000","last_clean":"2024-03-21T08:56:00.082401+0000","last_became_active":"2024-03-21T08:56:00.077347+0000","last_became_peered":"2024-03-21T08:56:00.077347+0000","last_unstale":"2024-03-21T08:56:00.082401+0000","last_undegraded":"2024-03-21T08:56:00.082401+0000","last_fullsized":"2024-03-21T08:56:00.082401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:33.812992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072363+0000","last_change":"2024-03-21T08:55:07.532577+0000","last_active":"2024-03-21T08:56:00.072363+0000","last_peered":"2024-03-21T08:56:00.072363+0000","last_clean":"2024-03-21T08:56:00.072363+0000","last_became_active":"2024-03-21T08:55:07.529015+0000","last_became_peered":"2024-03-21T08:55:07.529015+0000","last_unstale":"2024-03-21T08:56:00.072363+0000","last_undegraded":"2024-03-21T08:56:00.072363+0000","last_fullsized":"2024-03-21T08:56:00.072363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:24.045732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069896+0000","last_change":"2024-03-21T08:55:06.238430+0000","last_active":"2024-03-21T08:56:00.069896+0000","last_peered":"2024-03-21T08:56:00.069896+0000","last_clean":"2024-03-21T08:56:00.069896+0000","last_became_active":"2024-03-21T08:55:06.238128+0000","last_became_peered":"2024-03-21T08:55:06.238128+0000","last_unstale":"2024-03-21T08:56:00.069896+0000","last_undegraded":"2024-03-21T08:56:00.069896+0000","last_fullsized":"2024-03-21T08:56:00.069896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:49.839546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066767+0000","last_change":"2024-03-21T08:55:07.540750+0000","last_active":"2024-03-21T08:56:00.066767+0000","last_peered":"2024-03-21T08:56:00.066767+0000","last_clean":"2024-03-21T08:56:00.066767+0000","last_became_active":"2024-03-21T08:55:07.540358+0000","last_became_peered":"2024-03-21T08:55:07.540358+0000","last_unstale":"2024-03-21T08:56:00.066767+0000","last_undegraded":"2024-03-21T08:56:00.066767+0000","last_fullsized":"2024-03-21T08:56:00.066767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:37.590268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:21.039098+0000","last_change":"2024-03-21T08:55:06.236387+0000","last_active":"2024-03-21T08:56:21.039098+0000","last_peered":"2024-03-21T08:56:21.039098+0000","last_clean":"2024-03-21T08:56:21.039098+0000","last_became_active":"2024-03-21T08:55:06.234603+0000","last_became_peered":"2024-03-21T08:55:06.234603+0000","last_unstale":"2024-03-21T08:56:21.039098+0000","last_undegraded":"2024-03-21T08:56:21.039098+0000","last_fullsized":"2024-03-21T08:56:21.039098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:10:28.921679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066553+0000","last_change":"2024-03-21T08:55:07.539011+0000","last_active":"2024-03-21T08:56:00.066553+0000","last_peered":"2024-03-21T08:56:00.066553+0000","last_clean":"2024-03-21T08:56:00.066553+0000","last_became_active":"2024-03-21T08:55:07.538694+0000","last_became_peered":"2024-03-21T08:55:07.538694+0000","last_unstale":"2024-03-21T08:56:00.066553+0000","last_undegraded":"2024-03-21T08:56:00.066553+0000","last_fullsized":"2024-03-21T08:56:00.066553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:52.746409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074466+0000","last_change":"2024-03-21T08:55:07.531605+0000","last_active":"2024-03-21T08:56:00.074466+0000","last_peered":"2024-03-21T08:56:00.074466+0000","last_clean":"2024-03-21T08:56:00.074466+0000","last_became_active":"2024-03-21T08:55:07.530207+0000","last_became_peered":"2024-03-21T08:55:07.530207+0000","last_unstale":"2024-03-21T08:56:00.074466+0000","last_undegraded":"2024-03-21T08:56:00.074466+0000","last_fullsized":"2024-03-21T08:56:00.074466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:58.805184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:21.038634+0000","last_change":"2024-03-21T08:55:06.228108+0000","last_active":"2024-03-21T08:56:21.038634+0000","last_peered":"2024-03-21T08:56:21.038634+0000","last_clean":"2024-03-21T08:56:21.038634+0000","last_became_active":"2024-03-21T08:55:06.227281+0000","last_became_peered":"2024-03-21T08:55:06.227281+0000","last_unstale":"2024-03-21T08:56:21.038634+0000","last_undegraded":"2024-03-21T08:56:21.038634+0000","last_fullsized":"2024-03-21T08:56:21.038634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:02.026031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392869+0000","last_change":"2024-03-21T08:55:07.531090+0000","last_active":"2024-03-21T08:56:00.392869+0000","last_peered":"2024-03-21T08:56:00.392869+0000","last_clean":"2024-03-21T08:56:00.392869+0000","last_became_active":"2024-03-21T08:55:07.530769+0000","last_became_peered":"2024-03-21T08:55:07.530769+0000","last_unstale":"2024-03-21T08:56:00.392869+0000","last_undegraded":"2024-03-21T08:56:00.392869+0000","last_fullsized":"2024-03-21T08:56:00.392869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:49:51.950743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074276+0000","last_change":"2024-03-21T08:55:06.228583+0000","last_active":"2024-03-21T08:56:00.074276+0000","last_peered":"2024-03-21T08:56:00.074276+0000","last_clean":"2024-03-21T08:56:00.074276+0000","last_became_active":"2024-03-21T08:55:06.228222+0000","last_became_peered":"2024-03-21T08:55:06.228222+0000","last_unstale":"2024-03-21T08:56:00.074276+0000","last_undegraded":"2024-03-21T08:56:00.074276+0000","last_fullsized":"2024-03-21T08:56:00.074276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:13:28.254775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070182+0000","last_change":"2024-03-21T08:53:52.956820+0000","last_active":"2024-03-21T08:56:00.070182+0000","last_peered":"2024-03-21T08:56:00.070182+0000","last_clean":"2024-03-21T08:56:00.070182+0000","last_became_active":"2024-03-21T08:53:52.956322+0000","last_became_peered":"2024-03-21T08:53:52.956322+0000","last_unstale":"2024-03-21T08:56:00.070182+0000","last_undegraded":"2024-03-21T08:56:00.070182+0000","last_fullsized":"2024-03-21T08:56:00.070182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_clean_scrub_stamp":"2024-03-21T08:50:27.622697+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:55.824313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068596+0000","last_change":"2024-03-21T08:55:07.529927+0000","last_active":"2024-03-21T08:56:00.068596+0000","last_peered":"2024-03-21T08:56:00.068596+0000","last_clean":"2024-03-21T08:56:00.068596+0000","last_became_active":"2024-03-21T08:55:07.529588+0000","last_became_peered":"2024-03-21T08:55:07.529588+0000","last_unstale":"2024-03-21T08:56:00.068596+0000","last_undegraded":"2024-03-21T08:56:00.068596+0000","last_fullsized":"2024-03-21T08:56:00.068596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:11:04.122448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070635+0000","last_change":"2024-03-21T08:55:06.236496+0000","last_active":"2024-03-21T08:56:00.070635+0000","last_peered":"2024-03-21T08:56:00.070635+0000","last_clean":"2024-03-21T08:56:00.070635+0000","last_became_active":"2024-03-21T08:55:06.234447+0000","last_became_peered":"2024-03-21T08:55:06.234447+0000","last_unstale":"2024-03-21T08:56:00.070635+0000","last_undegraded":"2024-03-21T08:56:00.070635+0000","last_fullsized":"2024-03-21T08:56:00.070635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:46:55.322330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076740+0000","last_change":"2024-03-21T08:55:06.232367+0000","last_active":"2024-03-21T08:56:00.076740+0000","last_peered":"2024-03-21T08:56:00.076740+0000","last_clean":"2024-03-21T08:56:00.076740+0000","last_became_active":"2024-03-21T08:55:06.229741+0000","last_became_peered":"2024-03-21T08:55:06.229741+0000","last_unstale":"2024-03-21T08:56:00.076740+0000","last_undegraded":"2024-03-21T08:56:00.076740+0000","last_fullsized":"2024-03-21T08:56:00.076740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:51.644926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068871+0000","last_change":"2024-03-21T08:55:07.527366+0000","last_active":"2024-03-21T08:56:00.068871+0000","last_peered":"2024-03-21T08:56:00.068871+0000","last_clean":"2024-03-21T08:56:00.068871+0000","last_became_active":"2024-03-21T08:55:07.527033+0000","last_became_peered":"2024-03-21T08:55:07.527033+0000","last_unstale":"2024-03-21T08:56:00.068871+0000","last_undegraded":"2024-03-21T08:56:00.068871+0000","last_fullsized":"2024-03-21T08:56:00.068871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:07:38.400820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070147+0000","last_change":"2024-03-21T08:55:06.234078+0000","last_active":"2024-03-21T08:56:00.070147+0000","last_peered":"2024-03-21T08:56:00.070147+0000","last_clean":"2024-03-21T08:56:00.070147+0000","last_became_active":"2024-03-21T08:55:06.233704+0000","last_became_peered":"2024-03-21T08:55:06.233704+0000","last_unstale":"2024-03-21T08:56:00.070147+0000","last_undegraded":"2024-03-21T08:56:00.070147+0000","last_fullsized":"2024-03-21T08:56:00.070147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:08:00.957142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393156+0000","last_change":"2024-03-21T08:55:07.529668+0000","last_active":"2024-03-21T08:56:00.393156+0000","last_peered":"2024-03-21T08:56:00.393156+0000","last_clean":"2024-03-21T08:56:00.393156+0000","last_became_active":"2024-03-21T08:55:07.528560+0000","last_became_peered":"2024-03-21T08:55:07.528560+0000","last_unstale":"2024-03-21T08:56:00.393156+0000","last_undegraded":"2024-03-21T08:56:00.393156+0000","last_fullsized":"2024-03-21T08:56:00.393156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:49.671107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074611+0000","last_change":"2024-03-21T08:55:06.241169+0000","last_active":"2024-03-21T08:56:00.074611+0000","last_peered":"2024-03-21T08:56:00.074611+0000","last_clean":"2024-03-21T08:56:00.074611+0000","last_became_active":"2024-03-21T08:55:06.240740+0000","last_became_peered":"2024-03-21T08:55:06.240740+0000","last_unstale":"2024-03-21T08:56:00.074611+0000","last_undegraded":"2024-03-21T08:56:00.074611+0000","last_fullsized":"2024-03-21T08:56:00.074611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:31.327983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068048+0000","last_change":"2024-03-21T08:55:07.544243+0000","last_active":"2024-03-21T08:56:00.068048+0000","last_peered":"2024-03-21T08:56:00.068048+0000","last_clean":"2024-03-21T08:56:00.068048+0000","last_became_active":"2024-03-21T08:55:07.543827+0000","last_became_peered":"2024-03-21T08:55:07.543827+0000","last_unstale":"2024-03-21T08:56:00.068048+0000","last_undegraded":"2024-03-21T08:56:00.068048+0000","last_fullsized":"2024-03-21T08:56:00.068048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:15:32.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.078546+0000","last_change":"2024-03-21T08:56:00.078546+0000","last_active":"2024-03-21T08:56:00.078546+0000","last_peered":"2024-03-21T08:56:00.078546+0000","last_clean":"2024-03-21T08:56:00.078546+0000","last_became_active":"2024-03-21T08:56:00.073617+0000","last_became_peered":"2024-03-21T08:56:00.073617+0000","last_unstale":"2024-03-21T08:56:00.078546+0000","last_undegraded":"2024-03-21T08:56:00.078546+0000","last_fullsized":"2024-03-21T08:56:00.078546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:55.511382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076027+0000","last_change":"2024-03-21T08:55:07.540974+0000","last_active":"2024-03-21T08:56:00.076027+0000","last_peered":"2024-03-21T08:56:00.076027+0000","last_clean":"2024-03-21T08:56:00.076027+0000","last_became_active":"2024-03-21T08:55:07.540246+0000","last_became_peered":"2024-03-21T08:55:07.540246+0000","last_unstale":"2024-03-21T08:56:00.076027+0000","last_undegraded":"2024-03-21T08:56:00.076027+0000","last_fullsized":"2024-03-21T08:56:00.076027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:14:27.255613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392795+0000","last_change":"2024-03-21T08:55:06.231544+0000","last_active":"2024-03-21T08:56:00.392795+0000","last_peered":"2024-03-21T08:56:00.392795+0000","last_clean":"2024-03-21T08:56:00.392795+0000","last_became_active":"2024-03-21T08:55:06.231238+0000","last_became_peered":"2024-03-21T08:55:06.231238+0000","last_unstale":"2024-03-21T08:56:00.392795+0000","last_undegraded":"2024-03-21T08:56:00.392795+0000","last_fullsized":"2024-03-21T08:56:00.392795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:34.831596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072473+0000","last_change":"2024-03-21T08:55:07.541189+0000","last_active":"2024-03-21T08:56:00.072473+0000","last_peered":"2024-03-21T08:56:00.072473+0000","last_clean":"2024-03-21T08:56:00.072473+0000","last_became_active":"2024-03-21T08:55:07.540819+0000","last_became_peered":"2024-03-21T08:55:07.540819+0000","last_unstale":"2024-03-21T08:56:00.072473+0000","last_undegraded":"2024-03-21T08:56:00.072473+0000","last_fullsized":"2024-03-21T08:56:00.072473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:21:43.879895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071366+0000","last_change":"2024-03-21T08:55:06.231164+0000","last_active":"2024-03-21T08:56:00.071366+0000","last_peered":"2024-03-21T08:56:00.071366+0000","last_clean":"2024-03-21T08:56:00.071366+0000","last_became_active":"2024-03-21T08:55:06.230782+0000","last_became_peered":"2024-03-21T08:55:06.230782+0000","last_unstale":"2024-03-21T08:56:00.071366+0000","last_undegraded":"2024-03-21T08:56:00.071366+0000","last_fullsized":"2024-03-21T08:56:00.071366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:26:56.744957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392895+0000","last_change":"2024-03-21T08:55:07.544379+0000","last_active":"2024-03-21T08:56:00.392895+0000","last_peered":"2024-03-21T08:56:00.392895+0000","last_clean":"2024-03-21T08:56:00.392895+0000","last_became_active":"2024-03-21T08:55:07.541643+0000","last_became_peered":"2024-03-21T08:55:07.541643+0000","last_unstale":"2024-03-21T08:56:00.392895+0000","last_undegraded":"2024-03-21T08:56:00.392895+0000","last_fullsized":"2024-03-21T08:56:00.392895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:54:19.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076493+0000","last_change":"2024-03-21T08:55:06.229859+0000","last_active":"2024-03-21T08:56:00.076493+0000","last_peered":"2024-03-21T08:56:00.076493+0000","last_clean":"2024-03-21T08:56:00.076493+0000","last_became_active":"2024-03-21T08:55:06.229532+0000","last_became_peered":"2024-03-21T08:55:06.229532+0000","last_unstale":"2024-03-21T08:56:00.076493+0000","last_undegraded":"2024-03-21T08:56:00.076493+0000","last_fullsized":"2024-03-21T08:56:00.076493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:39.615417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071000+0000","last_change":"2024-03-21T08:55:07.530619+0000","last_active":"2024-03-21T08:56:00.071000+0000","last_peered":"2024-03-21T08:56:00.071000+0000","last_clean":"2024-03-21T08:56:00.071000+0000","last_became_active":"2024-03-21T08:55:07.530293+0000","last_became_peered":"2024-03-21T08:55:07.530293+0000","last_unstale":"2024-03-21T08:56:00.071000+0000","last_undegraded":"2024-03-21T08:56:00.071000+0000","last_fullsized":"2024-03-21T08:56:00.071000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:44:56.242621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:20.037738+0000","last_change":"2024-03-21T08:55:06.241581+0000","last_active":"2024-03-21T08:56:20.037738+0000","last_peered":"2024-03-21T08:56:20.037738+0000","last_clean":"2024-03-21T08:56:20.037738+0000","last_became_active":"2024-03-21T08:55:06.241162+0000","last_became_peered":"2024-03-21T08:55:06.241162+0000","last_unstale":"2024-03-21T08:56:20.037738+0000","last_undegraded":"2024-03-21T08:56:20.037738+0000","last_fullsized":"2024-03-21T08:56:20.037738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:15.231840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070496+0000","last_change":"2024-03-21T08:55:07.532424+0000","last_active":"2024-03-21T08:56:00.070496+0000","last_peered":"2024-03-21T08:56:00.070496+0000","last_clean":"2024-03-21T08:56:00.070496+0000","last_became_active":"2024-03-21T08:55:07.532062+0000","last_became_peered":"2024-03-21T08:55:07.532062+0000","last_unstale":"2024-03-21T08:56:00.070496+0000","last_undegraded":"2024-03-21T08:56:00.070496+0000","last_fullsized":"2024-03-21T08:56:00.070496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:44:07.970387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074454+0000","last_change":"2024-03-21T08:55:06.232191+0000","last_active":"2024-03-21T08:56:00.074454+0000","last_peered":"2024-03-21T08:56:00.074454+0000","last_clean":"2024-03-21T08:56:00.074454+0000","last_became_active":"2024-03-21T08:55:06.231880+0000","last_became_peered":"2024-03-21T08:55:06.231880+0000","last_unstale":"2024-03-21T08:56:00.074454+0000","last_undegraded":"2024-03-21T08:56:00.074454+0000","last_fullsized":"2024-03-21T08:56:00.074454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:35.434959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070331+0000","last_change":"2024-03-21T08:55:07.538588+0000","last_active":"2024-03-21T08:56:00.070331+0000","last_peered":"2024-03-21T08:56:00.070331+0000","last_clean":"2024-03-21T08:56:00.070331+0000","last_became_active":"2024-03-21T08:55:07.538268+0000","last_became_peered":"2024-03-21T08:55:07.538268+0000","last_unstale":"2024-03-21T08:56:00.070331+0000","last_undegraded":"2024-03-21T08:56:00.070331+0000","last_fullsized":"2024-03-21T08:56:00.070331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:21.591094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068839+0000","last_change":"2024-03-21T08:55:06.228166+0000","last_active":"2024-03-21T08:56:00.068839+0000","last_peered":"2024-03-21T08:56:00.068839+0000","last_clean":"2024-03-21T08:56:00.068839+0000","last_became_active":"2024-03-21T08:55:06.227852+0000","last_became_peered":"2024-03-21T08:55:06.227852+0000","last_unstale":"2024-03-21T08:56:00.068839+0000","last_undegraded":"2024-03-21T08:56:00.068839+0000","last_fullsized":"2024-03-21T08:56:00.068839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:12.471809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070562+0000","last_change":"2024-03-21T08:55:07.536854+0000","last_active":"2024-03-21T08:56:00.070562+0000","last_peered":"2024-03-21T08:56:00.070562+0000","last_clean":"2024-03-21T08:56:00.070562+0000","last_became_active":"2024-03-21T08:55:07.536241+0000","last_became_peered":"2024-03-21T08:55:07.536241+0000","last_unstale":"2024-03-21T08:56:00.070562+0000","last_undegraded":"2024-03-21T08:56:00.070562+0000","last_fullsized":"2024-03-21T08:56:00.070562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:21.060162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072594+0000","last_change":"2024-03-21T08:55:07.528084+0000","last_active":"2024-03-21T08:56:00.072594+0000","last_peered":"2024-03-21T08:56:00.072594+0000","last_clean":"2024-03-21T08:56:00.072594+0000","last_became_active":"2024-03-21T08:55:07.527675+0000","last_became_peered":"2024-03-21T08:55:07.527675+0000","last_unstale":"2024-03-21T08:56:00.072594+0000","last_undegraded":"2024-03-21T08:56:00.072594+0000","last_fullsized":"2024-03-21T08:56:00.072594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:47:21.646078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071316+0000","last_change":"2024-03-21T08:55:07.542384+0000","last_active":"2024-03-21T08:56:00.071316+0000","last_peered":"2024-03-21T08:56:00.071316+0000","last_clean":"2024-03-21T08:56:00.071316+0000","last_became_active":"2024-03-21T08:55:07.541956+0000","last_became_peered":"2024-03-21T08:55:07.541956+0000","last_unstale":"2024-03-21T08:56:00.071316+0000","last_undegraded":"2024-03-21T08:56:00.071316+0000","last_fullsized":"2024-03-21T08:56:00.071316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:54:47.093433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071248+0000","last_change":"2024-03-21T08:55:07.536226+0000","last_active":"2024-03-21T08:56:00.071248+0000","last_peered":"2024-03-21T08:56:00.071248+0000","last_clean":"2024-03-21T08:56:00.071248+0000","last_became_active":"2024-03-21T08:55:07.535711+0000","last_became_peered":"2024-03-21T08:55:07.535711+0000","last_unstale":"2024-03-21T08:56:00.071248+0000","last_undegraded":"2024-03-21T08:56:00.071248+0000","last_fullsized":"2024-03-21T08:56:00.071248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:37:29.252051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068051+0000","last_change":"2024-03-21T08:55:07.536440+0000","last_active":"2024-03-21T08:56:00.068051+0000","last_peered":"2024-03-21T08:56:00.068051+0000","last_clean":"2024-03-21T08:56:00.068051+0000","last_became_active":"2024-03-21T08:55:07.536076+0000","last_became_peered":"2024-03-21T08:55:07.536076+0000","last_unstale":"2024-03-21T08:56:00.068051+0000","last_undegraded":"2024-03-21T08:56:00.068051+0000","last_fullsized":"2024-03-21T08:56:00.068051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:26:30.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068879+0000","last_change":"2024-03-21T08:55:07.542634+0000","last_active":"2024-03-21T08:56:00.068879+0000","last_peered":"2024-03-21T08:56:00.068879+0000","last_clean":"2024-03-21T08:56:00.068879+0000","last_became_active":"2024-03-21T08:55:07.542317+0000","last_became_peered":"2024-03-21T08:55:07.542317+0000","last_unstale":"2024-03-21T08:56:00.068879+0000","last_undegraded":"2024-03-21T08:56:00.068879+0000","last_fullsized":"2024-03-21T08:56:00.068879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:31.392606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072408+0000","last_change":"2024-03-21T08:55:07.537660+0000","last_active":"2024-03-21T08:56:00.072408+0000","last_peered":"2024-03-21T08:56:00.072408+0000","last_clean":"2024-03-21T08:56:00.072408+0000","last_became_active":"2024-03-21T08:55:07.537205+0000","last_became_peered":"2024-03-21T08:55:07.537205+0000","last_unstale":"2024-03-21T08:56:00.072408+0000","last_undegraded":"2024-03-21T08:56:00.072408+0000","last_fullsized":"2024-03-21T08:56:00.072408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:39:59.336323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071576+0000","last_change":"2024-03-21T08:55:07.530140+0000","last_active":"2024-03-21T08:56:00.071576+0000","last_peered":"2024-03-21T08:56:00.071576+0000","last_clean":"2024-03-21T08:56:00.071576+0000","last_became_active":"2024-03-21T08:55:07.529373+0000","last_became_peered":"2024-03-21T08:55:07.529373+0000","last_unstale":"2024-03-21T08:56:00.071576+0000","last_undegraded":"2024-03-21T08:56:00.071576+0000","last_fullsized":"2024-03-21T08:56:00.071576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:37.939733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393067+0000","last_change":"2024-03-21T08:55:07.529218+0000","last_active":"2024-03-21T08:56:00.393067+0000","last_peered":"2024-03-21T08:56:00.393067+0000","last_clean":"2024-03-21T08:56:00.393067+0000","last_became_active":"2024-03-21T08:55:07.528898+0000","last_became_peered":"2024-03-21T08:55:07.528898+0000","last_unstale":"2024-03-21T08:56:00.393067+0000","last_undegraded":"2024-03-21T08:56:00.393067+0000","last_fullsized":"2024-03-21T08:56:00.393067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:49.392688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074560+0000","last_change":"2024-03-21T08:55:07.531920+0000","last_active":"2024-03-21T08:56:00.074560+0000","last_peered":"2024-03-21T08:56:00.074560+0000","last_clean":"2024-03-21T08:56:00.074560+0000","last_became_active":"2024-03-21T08:55:07.531266+0000","last_became_peered":"2024-03-21T08:55:07.531266+0000","last_unstale":"2024-03-21T08:56:00.074560+0000","last_undegraded":"2024-03-21T08:56:00.074560+0000","last_fullsized":"2024-03-21T08:56:00.074560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:41:19.838525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392678+0000","last_change":"2024-03-21T08:55:07.530658+0000","last_active":"2024-03-21T08:56:00.392678+0000","last_peered":"2024-03-21T08:56:00.392678+0000","last_clean":"2024-03-21T08:56:00.392678+0000","last_became_active":"2024-03-21T08:55:07.529933+0000","last_became_peered":"2024-03-21T08:55:07.529933+0000","last_unstale":"2024-03-21T08:56:00.392678+0000","last_undegraded":"2024-03-21T08:56:00.392678+0000","last_fullsized":"2024-03-21T08:56:00.392678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:56.698603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069429+0000","last_change":"2024-03-21T08:55:07.529151+0000","last_active":"2024-03-21T08:56:00.069429+0000","last_peered":"2024-03-21T08:56:00.069429+0000","last_clean":"2024-03-21T08:56:00.069429+0000","last_became_active":"2024-03-21T08:55:07.528511+0000","last_became_peered":"2024-03-21T08:55:07.528511+0000","last_unstale":"2024-03-21T08:56:00.069429+0000","last_undegraded":"2024-03-21T08:56:00.069429+0000","last_fullsized":"2024-03-21T08:56:00.069429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:40:27.720391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069226+0000","last_change":"2024-03-21T08:55:07.541519+0000","last_active":"2024-03-21T08:56:00.069226+0000","last_peered":"2024-03-21T08:56:00.069226+0000","last_clean":"2024-03-21T08:56:00.069226+0000","last_became_active":"2024-03-21T08:55:07.541171+0000","last_became_peered":"2024-03-21T08:55:07.541171+0000","last_unstale":"2024-03-21T08:56:00.069226+0000","last_undegraded":"2024-03-21T08:56:00.069226+0000","last_fullsized":"2024-03-21T08:56:00.069226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:10:36.249778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076625+0000","last_change":"2024-03-21T08:56:00.076625+0000","last_active":"2024-03-21T08:56:00.076625+0000","last_peered":"2024-03-21T08:56:00.076625+0000","last_clean":"2024-03-21T08:56:00.076625+0000","last_became_active":"2024-03-21T08:56:00.076266+0000","last_became_peered":"2024-03-21T08:56:00.076266+0000","last_unstale":"2024-03-21T08:56:00.076625+0000","last_undegraded":"2024-03-21T08:56:00.076625+0000","last_fullsized":"2024-03-21T08:56:00.076625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:55.701867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280365,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67859,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":732,"num_read_kb":437,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":203577,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351004,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":461228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612641,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1113672,"data_compressed":37511,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776167,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":461136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037803,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":461492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1054032,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":460853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574927,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":522812,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738452,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":462192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":523032,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182469,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":460678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1053319,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62798,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63018,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63173,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:56:25.085 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph pg dump --format=json 2024-03-21T08:56:25.341 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:25.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:56:25.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:25.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:56:25.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:56:25.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:56:25.630 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:25.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:56:25.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: pgmap v331: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:25.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:56:25.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:56:25.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:56:27.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 -- 172.21.15.87:0/2007267792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f0107550 msgr2=0x7fa9f010f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:27.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2007267792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f0107550 0x7fa9f010f0d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b0d0 tx=0x7fa9e800b3e0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 -- 172.21.15.87:0/2007267792 shutdown_connections 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2007267792 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f0107550 0x7fa9f010f0d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2007267792 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9f0100d30 0x7fa9f0101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2007267792 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f01016e0 0x7fa9f0106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 -- 172.21.15.87:0/2007267792 >> 172.21.15.87:0/2007267792 conn(0x7fa9f00763f0 msgr2=0x7fa9f0076800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 -- 172.21.15.87:0/2007267792 shutdown_connections 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.055+0000 7fa9f7241700 1 -- 172.21.15.87:0/2007267792 wait complete. 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 Processor -- start 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 -- start start 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9f0100d30 0x7fa9f0101cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 0x7fa9f01021f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:27.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f0107550 0x7fa9f0102730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0112510 con 0x7fa9f01016e0 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0112390 con 0x7fa9f0107550 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f7241700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f0112690 con 0x7fa9f0100d30 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9e7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 0x7fa9f01021f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f57de700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f0107550 0x7fa9f0102730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9e7fff700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 0x7fa9f01021f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:49494/0 (socket says 172.21.15.87:49494) 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9e7fff700 1 -- 172.21.15.87:0/2565517745 learned_addr learned my addr 172.21.15.87:0/2565517745 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f4fdd700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9f0100d30 0x7fa9f0101cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f57de700 1 -- 172.21.15.87:0/2565517745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9f0100d30 msgr2=0x7fa9f0101cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:27.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f57de700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9f0100d30 0x7fa9f0101cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f57de700 1 -- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 msgr2=0x7fa9f01021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f57de700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 0x7fa9f01021f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9f57de700 1 -- 172.21.15.87:0/2565517745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f01a6e90 con 0x7fa9f0107550 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.056+0000 7fa9e7fff700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 0x7fa9f01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.057+0000 7fa9f57de700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f0107550 0x7fa9f0102730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b770 tx=0x7fa9e8007a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.057+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e8021020 con 0x7fa9f0107550 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.057+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f01a7120 con 0x7fa9f0107550 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.057+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa9e8004c50 con 0x7fa9f0107550 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.057+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e80052b0 con 0x7fa9f0107550 2024-03-21T08:56:27.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.057+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f01a7620 con 0x7fa9f0107550 2024-03-21T08:56:27.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.058+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f010a790 con 0x7fa9f0107550 2024-03-21T08:56:27.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.059+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 17) v1 ==== 137905+0+0 (secure 0 0 0) 0x7fa9e8024020 con 0x7fa9f0107550 2024-03-21T08:56:27.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.059+0000 7fa9e5ffb700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9d80a3040 0x7fa9d80a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:27.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.059+0000 7fa9f4fdd700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9d80a3040 0x7fa9d80a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:27.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.059+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9e800f020 con 0x7fa9f0107550 2024-03-21T08:56:27.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.060+0000 7fa9f4fdd700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9d80a3040 0x7fa9d80a5500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009c60 tx=0x7fa9ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:27.058 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.063+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa9e801de20 con 0x7fa9f0107550 2024-03-21T08:56:27.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.253+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa9f00625d0 con 0x7fa9d80a3040 2024-03-21T08:56:27.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.260+0000 7fa9e5ffb700 1 -- 172.21.15.87:0/2565517745 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+258981 (secure 0 0 0) 0x7fa9f00625d0 con 0x7fa9d80a3040 2024-03-21T08:56:27.259 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:27.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9d80a3040 msgr2=0x7fa9d80a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:27.261 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9d80a3040 0x7fa9d80a5500 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec009c60 tx=0x7fa9ec009260 comp rx=0 tx=0).stop 2024-03-21T08:56:27.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f0107550 msgr2=0x7fa9f0102730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:27.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f0107550 0x7fa9f0102730 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9e800b770 tx=0x7fa9e8007a20 comp rx=0 tx=0).stop 2024-03-21T08:56:27.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 shutdown_connections 2024-03-21T08:56:27.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9d80a3040 0x7fa9d80a5500 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9f0107550 0x7fa9f0102730 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.262 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9f0100d30 0x7fa9f0101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 --2- 172.21.15.87:0/2565517745 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa9f01016e0 0x7fa9f01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 >> 172.21.15.87:0/2565517745 conn(0x7fa9f00763f0 msgr2=0x7fa9f00feae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 shutdown_connections 2024-03-21T08:56:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:27.264+0000 7fa9f7241700 1 -- 172.21.15.87:0/2565517745 wait complete. 2024-03-21T08:56:27.263 INFO:teuthology.orchestra.run.smithi087.stderr:dumped all 2024-03-21T08:56:27.999 INFO:teuthology.orchestra.run.smithi087.stdout:{"pg_ready":true,"pg_map":{"version":332,"stamp":"2024-03-21T08:56:26.873356+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658461,"num_objects":102,"num_object_clones":0,"num_object_copies":306,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":883,"num_read_kb":566,"num_write":419,"num_write_kb":3072,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180758,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":344,"ondisk_log_size":344,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356920,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584904,"statfs":{"total":1151940427776,"available":1151574941696,"internally_reserved":0,"allocated":12128256,"data_stored":7495551,"data_compressed":112533,"data_compressed_allocated":147456,"data_compressed_original":1794048,"omap_allocated":19143,"internal_metadata":353154361},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":100,"num_read_kb":50,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003197"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072875+0000","last_change":"2024-03-21T08:55:07.542089+0000","last_active":"2024-03-21T08:56:00.072875+0000","last_peered":"2024-03-21T08:56:00.072875+0000","last_clean":"2024-03-21T08:56:00.072875+0000","last_became_active":"2024-03-21T08:55:07.541790+0000","last_became_peered":"2024-03-21T08:55:07.541790+0000","last_unstale":"2024-03-21T08:56:00.072875+0000","last_undegraded":"2024-03-21T08:56:00.072875+0000","last_fullsized":"2024-03-21T08:56:00.072875+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:57:10.863345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068003+0000","last_change":"2024-03-21T08:55:07.543892+0000","last_active":"2024-03-21T08:56:00.068003+0000","last_peered":"2024-03-21T08:56:00.068003+0000","last_clean":"2024-03-21T08:56:00.068003+0000","last_became_active":"2024-03-21T08:55:07.541230+0000","last_became_peered":"2024-03-21T08:55:07.541230+0000","last_unstale":"2024-03-21T08:56:00.068003+0000","last_undegraded":"2024-03-21T08:56:00.068003+0000","last_fullsized":"2024-03-21T08:56:00.068003+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:52:35.545762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068237+0000","last_change":"2024-03-21T08:55:07.537095+0000","last_active":"2024-03-21T08:56:00.068237+0000","last_peered":"2024-03-21T08:56:00.068237+0000","last_clean":"2024-03-21T08:56:00.068237+0000","last_became_active":"2024-03-21T08:55:07.536784+0000","last_became_peered":"2024-03-21T08:55:07.536784+0000","last_unstale":"2024-03-21T08:56:00.068237+0000","last_undegraded":"2024-03-21T08:56:00.068237+0000","last_fullsized":"2024-03-21T08:56:00.068237+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:42:09.373813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074809+0000","last_change":"2024-03-21T08:55:07.533330+0000","last_active":"2024-03-21T08:56:00.074809+0000","last_peered":"2024-03-21T08:56:00.074809+0000","last_clean":"2024-03-21T08:56:00.074809+0000","last_became_active":"2024-03-21T08:55:07.532840+0000","last_became_peered":"2024-03-21T08:55:07.532840+0000","last_unstale":"2024-03-21T08:56:00.074809+0000","last_undegraded":"2024-03-21T08:56:00.074809+0000","last_fullsized":"2024-03-21T08:56:00.074809+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:52:36.565830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076296+0000","last_change":"2024-03-21T08:55:07.549259+0000","last_active":"2024-03-21T08:56:00.076296+0000","last_peered":"2024-03-21T08:56:00.076296+0000","last_clean":"2024-03-21T08:56:00.076296+0000","last_became_active":"2024-03-21T08:55:07.548161+0000","last_became_peered":"2024-03-21T08:55:07.548161+0000","last_unstale":"2024-03-21T08:56:00.076296+0000","last_undegraded":"2024-03-21T08:56:00.076296+0000","last_fullsized":"2024-03-21T08:56:00.076296+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:54:12.803360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066466+0000","last_change":"2024-03-21T08:55:07.530950+0000","last_active":"2024-03-21T08:56:00.066466+0000","last_peered":"2024-03-21T08:56:00.066466+0000","last_clean":"2024-03-21T08:56:00.066466+0000","last_became_active":"2024-03-21T08:55:07.530582+0000","last_became_peered":"2024-03-21T08:55:07.530582+0000","last_unstale":"2024-03-21T08:56:00.066466+0000","last_undegraded":"2024-03-21T08:56:00.066466+0000","last_fullsized":"2024-03-21T08:56:00.066466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:56:12.380733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070232+0000","last_change":"2024-03-21T08:55:07.535591+0000","last_active":"2024-03-21T08:56:00.070232+0000","last_peered":"2024-03-21T08:56:00.070232+0000","last_clean":"2024-03-21T08:56:00.070232+0000","last_became_active":"2024-03-21T08:55:07.535257+0000","last_became_peered":"2024-03-21T08:55:07.535257+0000","last_unstale":"2024-03-21T08:56:00.070232+0000","last_undegraded":"2024-03-21T08:56:00.070232+0000","last_fullsized":"2024-03-21T08:56:00.070232+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:27:39.383653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392635+0000","last_change":"2024-03-21T08:55:07.532165+0000","last_active":"2024-03-21T08:56:00.392635+0000","last_peered":"2024-03-21T08:56:00.392635+0000","last_clean":"2024-03-21T08:56:00.392635+0000","last_became_active":"2024-03-21T08:55:07.531822+0000","last_became_peered":"2024-03-21T08:55:07.531822+0000","last_unstale":"2024-03-21T08:56:00.392635+0000","last_undegraded":"2024-03-21T08:56:00.392635+0000","last_fullsized":"2024-03-21T08:56:00.392635+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:01:56.245424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.077869+0000","last_change":"2024-03-21T08:56:00.077869+0000","last_active":"2024-03-21T08:56:00.077869+0000","last_peered":"2024-03-21T08:56:00.077869+0000","last_clean":"2024-03-21T08:56:00.077869+0000","last_became_active":"2024-03-21T08:56:00.077541+0000","last_became_peered":"2024-03-21T08:56:00.077541+0000","last_unstale":"2024-03-21T08:56:00.077869+0000","last_undegraded":"2024-03-21T08:56:00.077869+0000","last_fullsized":"2024-03-21T08:56:00.077869+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:23.227887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,11],"acting":[1,8,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066888+0000","last_change":"2024-03-21T08:55:07.532227+0000","last_active":"2024-03-21T08:56:00.066888+0000","last_peered":"2024-03-21T08:56:00.066888+0000","last_clean":"2024-03-21T08:56:00.066888+0000","last_became_active":"2024-03-21T08:55:07.531349+0000","last_became_peered":"2024-03-21T08:55:07.531349+0000","last_unstale":"2024-03-21T08:56:00.066888+0000","last_undegraded":"2024-03-21T08:56:00.066888+0000","last_fullsized":"2024-03-21T08:56:00.066888+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:07:40.318544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070379+0000","last_change":"2024-03-21T08:55:07.533531+0000","last_active":"2024-03-21T08:56:00.070379+0000","last_peered":"2024-03-21T08:56:00.070379+0000","last_clean":"2024-03-21T08:56:00.070379+0000","last_became_active":"2024-03-21T08:55:07.533166+0000","last_became_peered":"2024-03-21T08:55:07.533166+0000","last_unstale":"2024-03-21T08:56:00.070379+0000","last_undegraded":"2024-03-21T08:56:00.070379+0000","last_fullsized":"2024-03-21T08:56:00.070379+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:19:24.912748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074799+0000","last_change":"2024-03-21T08:55:07.533643+0000","last_active":"2024-03-21T08:56:00.074799+0000","last_peered":"2024-03-21T08:56:00.074799+0000","last_clean":"2024-03-21T08:56:00.074799+0000","last_became_active":"2024-03-21T08:55:07.533262+0000","last_became_peered":"2024-03-21T08:55:07.533262+0000","last_unstale":"2024-03-21T08:56:00.074799+0000","last_undegraded":"2024-03-21T08:56:00.074799+0000","last_fullsized":"2024-03-21T08:56:00.074799+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:51:07.641225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069129+0000","last_change":"2024-03-21T08:55:07.524670+0000","last_active":"2024-03-21T08:56:00.069129+0000","last_peered":"2024-03-21T08:56:00.069129+0000","last_clean":"2024-03-21T08:56:00.069129+0000","last_became_active":"2024-03-21T08:55:07.524295+0000","last_became_peered":"2024-03-21T08:55:07.524295+0000","last_unstale":"2024-03-21T08:56:00.069129+0000","last_undegraded":"2024-03-21T08:56:00.069129+0000","last_fullsized":"2024-03-21T08:56:00.069129+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:38:07.506696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068383+0000","last_change":"2024-03-21T08:55:07.534401+0000","last_active":"2024-03-21T08:56:00.068383+0000","last_peered":"2024-03-21T08:56:00.068383+0000","last_clean":"2024-03-21T08:56:00.068383+0000","last_became_active":"2024-03-21T08:55:07.534056+0000","last_became_peered":"2024-03-21T08:55:07.534056+0000","last_unstale":"2024-03-21T08:56:00.068383+0000","last_undegraded":"2024-03-21T08:56:00.068383+0000","last_fullsized":"2024-03-21T08:56:00.068383+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:57:55.943564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.067782+0000","last_change":"2024-03-21T08:55:07.535915+0000","last_active":"2024-03-21T08:56:00.067782+0000","last_peered":"2024-03-21T08:56:00.067782+0000","last_clean":"2024-03-21T08:56:00.067782+0000","last_became_active":"2024-03-21T08:55:07.535540+0000","last_became_peered":"2024-03-21T08:55:07.535540+0000","last_unstale":"2024-03-21T08:56:00.067782+0000","last_undegraded":"2024-03-21T08:56:00.067782+0000","last_fullsized":"2024-03-21T08:56:00.067782+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:43:30.916226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069239+0000","last_change":"2024-03-21T08:55:07.541894+0000","last_active":"2024-03-21T08:56:00.069239+0000","last_peered":"2024-03-21T08:56:00.069239+0000","last_clean":"2024-03-21T08:56:00.069239+0000","last_became_active":"2024-03-21T08:55:07.541545+0000","last_became_peered":"2024-03-21T08:55:07.541545+0000","last_unstale":"2024-03-21T08:56:00.069239+0000","last_undegraded":"2024-03-21T08:56:00.069239+0000","last_fullsized":"2024-03-21T08:56:00.069239+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:11.090894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072516+0000","last_change":"2024-03-21T08:55:07.541228+0000","last_active":"2024-03-21T08:56:00.072516+0000","last_peered":"2024-03-21T08:56:00.072516+0000","last_clean":"2024-03-21T08:56:00.072516+0000","last_became_active":"2024-03-21T08:55:07.540836+0000","last_became_peered":"2024-03-21T08:55:07.540836+0000","last_unstale":"2024-03-21T08:56:00.072516+0000","last_undegraded":"2024-03-21T08:56:00.072516+0000","last_fullsized":"2024-03-21T08:56:00.072516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:34:09.491557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070181+0000","last_change":"2024-03-21T08:55:07.525778+0000","last_active":"2024-03-21T08:56:00.070181+0000","last_peered":"2024-03-21T08:56:00.070181+0000","last_clean":"2024-03-21T08:56:00.070181+0000","last_became_active":"2024-03-21T08:55:07.525469+0000","last_became_peered":"2024-03-21T08:55:07.525469+0000","last_unstale":"2024-03-21T08:56:00.070181+0000","last_undegraded":"2024-03-21T08:56:00.070181+0000","last_fullsized":"2024-03-21T08:56:00.070181+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:41:31.715570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071403+0000","last_change":"2024-03-21T08:55:07.529929+0000","last_active":"2024-03-21T08:56:00.071403+0000","last_peered":"2024-03-21T08:56:00.071403+0000","last_clean":"2024-03-21T08:56:00.071403+0000","last_became_active":"2024-03-21T08:55:07.529615+0000","last_became_peered":"2024-03-21T08:55:07.529615+0000","last_unstale":"2024-03-21T08:56:00.071403+0000","last_undegraded":"2024-03-21T08:56:00.071403+0000","last_fullsized":"2024-03-21T08:56:00.071403+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:42:19.543201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"86'13","reported_seq":105,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:23.040707+0000","last_change":"2024-03-21T08:55:06.239149+0000","last_active":"2024-03-21T08:56:23.040707+0000","last_peered":"2024-03-21T08:56:23.040707+0000","last_clean":"2024-03-21T08:56:23.040707+0000","last_became_active":"2024-03-21T08:55:06.238833+0000","last_became_peered":"2024-03-21T08:55:06.238833+0000","last_unstale":"2024-03-21T08:56:23.040707+0000","last_undegraded":"2024-03-21T08:56:23.040707+0000","last_fullsized":"2024-03-21T08:56:23.040707+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:28.779442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":73,"num_read_kb":73,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393075+0000","last_change":"2024-03-21T08:55:07.533596+0000","last_active":"2024-03-21T08:56:00.393075+0000","last_peered":"2024-03-21T08:56:00.393075+0000","last_clean":"2024-03-21T08:56:00.393075+0000","last_became_active":"2024-03-21T08:55:07.533249+0000","last_became_peered":"2024-03-21T08:55:07.533249+0000","last_unstale":"2024-03-21T08:56:00.393075+0000","last_undegraded":"2024-03-21T08:56:00.393075+0000","last_fullsized":"2024-03-21T08:56:00.393075+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:21:40.763491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"86'2","reported_seq":21,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393207+0000","last_change":"2024-03-21T08:55:06.227045+0000","last_active":"2024-03-21T08:56:00.393207+0000","last_peered":"2024-03-21T08:56:00.393207+0000","last_clean":"2024-03-21T08:56:00.393207+0000","last_became_active":"2024-03-21T08:55:06.226748+0000","last_became_peered":"2024-03-21T08:55:06.226748+0000","last_unstale":"2024-03-21T08:56:00.393207+0000","last_undegraded":"2024-03-21T08:56:00.393207+0000","last_fullsized":"2024-03-21T08:56:00.393207+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:41:37.677094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068191+0000","last_change":"2024-03-21T08:55:07.540024+0000","last_active":"2024-03-21T08:56:00.068191+0000","last_peered":"2024-03-21T08:56:00.068191+0000","last_clean":"2024-03-21T08:56:00.068191+0000","last_became_active":"2024-03-21T08:55:07.537574+0000","last_became_peered":"2024-03-21T08:55:07.537574+0000","last_unstale":"2024-03-21T08:56:00.068191+0000","last_undegraded":"2024-03-21T08:56:00.068191+0000","last_fullsized":"2024-03-21T08:56:00.068191+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:27:39.053045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"86'12","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076899+0000","last_change":"2024-03-21T08:55:06.237261+0000","last_active":"2024-03-21T08:56:00.076899+0000","last_peered":"2024-03-21T08:56:00.076899+0000","last_clean":"2024-03-21T08:56:00.076899+0000","last_became_active":"2024-03-21T08:55:06.236838+0000","last_became_peered":"2024-03-21T08:55:06.236838+0000","last_unstale":"2024-03-21T08:56:00.076899+0000","last_undegraded":"2024-03-21T08:56:00.076899+0000","last_fullsized":"2024-03-21T08:56:00.076899+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:49:59.881075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069339+0000","last_change":"2024-03-21T08:55:07.529159+0000","last_active":"2024-03-21T08:56:00.069339+0000","last_peered":"2024-03-21T08:56:00.069339+0000","last_clean":"2024-03-21T08:56:00.069339+0000","last_became_active":"2024-03-21T08:55:07.527260+0000","last_became_peered":"2024-03-21T08:55:07.527260+0000","last_unstale":"2024-03-21T08:56:00.069339+0000","last_undegraded":"2024-03-21T08:56:00.069339+0000","last_fullsized":"2024-03-21T08:56:00.069339+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:00:08.825873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076666+0000","last_change":"2024-03-21T08:55:06.232959+0000","last_active":"2024-03-21T08:56:00.076666+0000","last_peered":"2024-03-21T08:56:00.076666+0000","last_clean":"2024-03-21T08:56:00.076666+0000","last_became_active":"2024-03-21T08:55:06.232606+0000","last_became_peered":"2024-03-21T08:55:06.232606+0000","last_unstale":"2024-03-21T08:56:00.076666+0000","last_undegraded":"2024-03-21T08:56:00.076666+0000","last_fullsized":"2024-03-21T08:56:00.076666+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:27:07.480767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074544+0000","last_change":"2024-03-21T08:55:07.530711+0000","last_active":"2024-03-21T08:56:00.074544+0000","last_peered":"2024-03-21T08:56:00.074544+0000","last_clean":"2024-03-21T08:56:00.074544+0000","last_became_active":"2024-03-21T08:55:07.530374+0000","last_became_peered":"2024-03-21T08:55:07.530374+0000","last_unstale":"2024-03-21T08:56:00.074544+0000","last_undegraded":"2024-03-21T08:56:00.074544+0000","last_fullsized":"2024-03-21T08:56:00.074544+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:06.524172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"86'5","reported_seq":98,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:20.012199+0000","last_change":"2024-03-21T08:55:06.233934+0000","last_active":"2024-03-21T08:56:20.012199+0000","last_peered":"2024-03-21T08:56:20.012199+0000","last_clean":"2024-03-21T08:56:20.012199+0000","last_became_active":"2024-03-21T08:55:06.233618+0000","last_became_peered":"2024-03-21T08:55:06.233618+0000","last_unstale":"2024-03-21T08:56:20.012199+0000","last_undegraded":"2024-03-21T08:56:20.012199+0000","last_fullsized":"2024-03-21T08:56:20.012199+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:29:08.769883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":74,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076281+0000","last_change":"2024-03-21T08:55:07.546411+0000","last_active":"2024-03-21T08:56:00.076281+0000","last_peered":"2024-03-21T08:56:00.076281+0000","last_clean":"2024-03-21T08:56:00.076281+0000","last_became_active":"2024-03-21T08:55:07.546051+0000","last_became_peered":"2024-03-21T08:55:07.546051+0000","last_unstale":"2024-03-21T08:56:00.076281+0000","last_undegraded":"2024-03-21T08:56:00.076281+0000","last_fullsized":"2024-03-21T08:56:00.076281+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:04:29.400154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066447+0000","last_change":"2024-03-21T08:55:06.226791+0000","last_active":"2024-03-21T08:56:00.066447+0000","last_peered":"2024-03-21T08:56:00.066447+0000","last_clean":"2024-03-21T08:56:00.066447+0000","last_became_active":"2024-03-21T08:55:06.226438+0000","last_became_peered":"2024-03-21T08:55:06.226438+0000","last_unstale":"2024-03-21T08:56:00.066447+0000","last_undegraded":"2024-03-21T08:56:00.066447+0000","last_fullsized":"2024-03-21T08:56:00.066447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:15:03.732117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074475+0000","last_change":"2024-03-21T08:55:07.529623+0000","last_active":"2024-03-21T08:56:00.074475+0000","last_peered":"2024-03-21T08:56:00.074475+0000","last_clean":"2024-03-21T08:56:00.074475+0000","last_became_active":"2024-03-21T08:55:07.529293+0000","last_became_peered":"2024-03-21T08:55:07.529293+0000","last_unstale":"2024-03-21T08:56:00.074475+0000","last_undegraded":"2024-03-21T08:56:00.074475+0000","last_fullsized":"2024-03-21T08:56:00.074475+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:07:43.265861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068728+0000","last_change":"2024-03-21T08:55:06.236381+0000","last_active":"2024-03-21T08:56:00.068728+0000","last_peered":"2024-03-21T08:56:00.068728+0000","last_clean":"2024-03-21T08:56:00.068728+0000","last_became_active":"2024-03-21T08:55:06.236071+0000","last_became_peered":"2024-03-21T08:55:06.236071+0000","last_unstale":"2024-03-21T08:56:00.068728+0000","last_undegraded":"2024-03-21T08:56:00.068728+0000","last_fullsized":"2024-03-21T08:56:00.068728+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:49:13.338191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.073780+0000","last_change":"2024-03-21T08:56:00.073780+0000","last_active":"2024-03-21T08:56:00.073780+0000","last_peered":"2024-03-21T08:56:00.073780+0000","last_clean":"2024-03-21T08:56:00.073780+0000","last_became_active":"2024-03-21T08:56:00.073444+0000","last_became_peered":"2024-03-21T08:56:00.073444+0000","last_unstale":"2024-03-21T08:56:00.073780+0000","last_undegraded":"2024-03-21T08:56:00.073780+0000","last_fullsized":"2024-03-21T08:56:00.073780+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:46:25.618811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,6],"acting":[9,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074361+0000","last_change":"2024-03-21T08:55:06.233121+0000","last_active":"2024-03-21T08:56:00.074361+0000","last_peered":"2024-03-21T08:56:00.074361+0000","last_clean":"2024-03-21T08:56:00.074361+0000","last_became_active":"2024-03-21T08:55:06.232772+0000","last_became_peered":"2024-03-21T08:55:06.232772+0000","last_unstale":"2024-03-21T08:56:00.074361+0000","last_undegraded":"2024-03-21T08:56:00.074361+0000","last_fullsized":"2024-03-21T08:56:00.074361+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:37:05.203732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069830+0000","last_change":"2024-03-21T08:55:07.527483+0000","last_active":"2024-03-21T08:56:00.069830+0000","last_peered":"2024-03-21T08:56:00.069830+0000","last_clean":"2024-03-21T08:56:00.069830+0000","last_became_active":"2024-03-21T08:55:07.527038+0000","last_became_peered":"2024-03-21T08:55:07.527038+0000","last_unstale":"2024-03-21T08:56:00.069830+0000","last_undegraded":"2024-03-21T08:56:00.069830+0000","last_fullsized":"2024-03-21T08:56:00.069830+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:27:13.436766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"86'4","reported_seq":170,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:24.015674+0000","last_change":"2024-03-21T08:55:06.233677+0000","last_active":"2024-03-21T08:56:24.015674+0000","last_peered":"2024-03-21T08:56:24.015674+0000","last_clean":"2024-03-21T08:56:24.015674+0000","last_became_active":"2024-03-21T08:55:06.233312+0000","last_became_peered":"2024-03-21T08:55:06.233312+0000","last_unstale":"2024-03-21T08:56:24.015674+0000","last_undegraded":"2024-03-21T08:56:24.015674+0000","last_fullsized":"2024-03-21T08:56:24.015674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:51:53.894648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":147,"num_read_kb":147,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069835+0000","last_change":"2024-03-21T08:55:07.528174+0000","last_active":"2024-03-21T08:56:00.069835+0000","last_peered":"2024-03-21T08:56:00.069835+0000","last_clean":"2024-03-21T08:56:00.069835+0000","last_became_active":"2024-03-21T08:55:07.527313+0000","last_became_peered":"2024-03-21T08:55:07.527313+0000","last_unstale":"2024-03-21T08:56:00.069835+0000","last_undegraded":"2024-03-21T08:56:00.069835+0000","last_fullsized":"2024-03-21T08:56:00.069835+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:55:11.069071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070546+0000","last_change":"2024-03-21T08:55:06.237324+0000","last_active":"2024-03-21T08:56:00.070546+0000","last_peered":"2024-03-21T08:56:00.070546+0000","last_clean":"2024-03-21T08:56:00.070546+0000","last_became_active":"2024-03-21T08:55:06.235319+0000","last_became_peered":"2024-03-21T08:55:06.235319+0000","last_unstale":"2024-03-21T08:56:00.070546+0000","last_undegraded":"2024-03-21T08:56:00.070546+0000","last_fullsized":"2024-03-21T08:56:00.070546+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:05:32.872612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068119+0000","last_change":"2024-03-21T08:55:07.540552+0000","last_active":"2024-03-21T08:56:00.068119+0000","last_peered":"2024-03-21T08:56:00.068119+0000","last_clean":"2024-03-21T08:56:00.068119+0000","last_became_active":"2024-03-21T08:55:07.540211+0000","last_became_peered":"2024-03-21T08:55:07.540211+0000","last_unstale":"2024-03-21T08:56:00.068119+0000","last_undegraded":"2024-03-21T08:56:00.068119+0000","last_fullsized":"2024-03-21T08:56:00.068119+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:42.644523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074636+0000","last_change":"2024-03-21T08:55:06.228420+0000","last_active":"2024-03-21T08:56:00.074636+0000","last_peered":"2024-03-21T08:56:00.074636+0000","last_clean":"2024-03-21T08:56:00.074636+0000","last_became_active":"2024-03-21T08:55:06.228037+0000","last_became_peered":"2024-03-21T08:55:06.228037+0000","last_unstale":"2024-03-21T08:56:00.074636+0000","last_undegraded":"2024-03-21T08:56:00.074636+0000","last_fullsized":"2024-03-21T08:56:00.074636+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:55:22.437321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076329+0000","last_change":"2024-03-21T08:56:00.076329+0000","last_active":"2024-03-21T08:56:00.076329+0000","last_peered":"2024-03-21T08:56:00.076329+0000","last_clean":"2024-03-21T08:56:00.076329+0000","last_became_active":"2024-03-21T08:56:00.076092+0000","last_became_peered":"2024-03-21T08:56:00.076092+0000","last_unstale":"2024-03-21T08:56:00.076329+0000","last_undegraded":"2024-03-21T08:56:00.076329+0000","last_fullsized":"2024-03-21T08:56:00.076329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:18:04.674021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,6],"acting":[1,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"86'30","reported_seq":208,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:24.015817+0000","last_change":"2024-03-21T08:55:06.232681+0000","last_active":"2024-03-21T08:56:24.015817+0000","last_peered":"2024-03-21T08:56:24.015817+0000","last_clean":"2024-03-21T08:56:24.015817+0000","last_became_active":"2024-03-21T08:55:06.232109+0000","last_became_peered":"2024-03-21T08:55:06.232109+0000","last_unstale":"2024-03-21T08:56:24.015817+0000","last_undegraded":"2024-03-21T08:56:24.015817+0000","last_fullsized":"2024-03-21T08:56:24.015817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:59:11.218106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":61046,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":159,"num_read_kb":139,"num_write":32,"num_write_kb":69,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076644+0000","last_change":"2024-03-21T08:55:07.546464+0000","last_active":"2024-03-21T08:56:00.076644+0000","last_peered":"2024-03-21T08:56:00.076644+0000","last_clean":"2024-03-21T08:56:00.076644+0000","last_became_active":"2024-03-21T08:55:07.546111+0000","last_became_peered":"2024-03-21T08:55:07.546111+0000","last_unstale":"2024-03-21T08:56:00.076644+0000","last_undegraded":"2024-03-21T08:56:00.076644+0000","last_fullsized":"2024-03-21T08:56:00.076644+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:57:12.651728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"86'9","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071398+0000","last_change":"2024-03-21T08:55:06.233157+0000","last_active":"2024-03-21T08:56:00.071398+0000","last_peered":"2024-03-21T08:56:00.071398+0000","last_clean":"2024-03-21T08:56:00.071398+0000","last_became_active":"2024-03-21T08:55:06.232302+0000","last_became_peered":"2024-03-21T08:55:06.232302+0000","last_unstale":"2024-03-21T08:56:00.071398+0000","last_undegraded":"2024-03-21T08:56:00.071398+0000","last_fullsized":"2024-03-21T08:56:00.071398+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:09:47.550454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068774+0000","last_change":"2024-03-21T08:55:07.531265+0000","last_active":"2024-03-21T08:56:00.068774+0000","last_peered":"2024-03-21T08:56:00.068774+0000","last_clean":"2024-03-21T08:56:00.068774+0000","last_became_active":"2024-03-21T08:55:07.530913+0000","last_became_peered":"2024-03-21T08:55:07.530913+0000","last_unstale":"2024-03-21T08:56:00.068774+0000","last_undegraded":"2024-03-21T08:56:00.068774+0000","last_fullsized":"2024-03-21T08:56:00.068774+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:38:34.171463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"86'5","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074401+0000","last_change":"2024-03-21T08:55:06.233411+0000","last_active":"2024-03-21T08:56:00.074401+0000","last_peered":"2024-03-21T08:56:00.074401+0000","last_clean":"2024-03-21T08:56:00.074401+0000","last_became_active":"2024-03-21T08:55:06.233031+0000","last_became_peered":"2024-03-21T08:55:06.233031+0000","last_unstale":"2024-03-21T08:56:00.074401+0000","last_undegraded":"2024-03-21T08:56:00.074401+0000","last_fullsized":"2024-03-21T08:56:00.074401+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:52:07.258184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072534+0000","last_change":"2024-03-21T08:56:00.072534+0000","last_active":"2024-03-21T08:56:00.072534+0000","last_peered":"2024-03-21T08:56:00.072534+0000","last_clean":"2024-03-21T08:56:00.072534+0000","last_became_active":"2024-03-21T08:56:00.072138+0000","last_became_peered":"2024-03-21T08:56:00.072138+0000","last_unstale":"2024-03-21T08:56:00.072534+0000","last_undegraded":"2024-03-21T08:56:00.072534+0000","last_fullsized":"2024-03-21T08:56:00.072534+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:38:19.550293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,6],"acting":[7,9,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"86'6","reported_seq":96,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:20.038147+0000","last_change":"2024-03-21T08:55:06.230587+0000","last_active":"2024-03-21T08:56:20.038147+0000","last_peered":"2024-03-21T08:56:20.038147+0000","last_clean":"2024-03-21T08:56:20.038147+0000","last_became_active":"2024-03-21T08:55:06.230201+0000","last_became_peered":"2024-03-21T08:55:06.230201+0000","last_unstale":"2024-03-21T08:56:20.038147+0000","last_undegraded":"2024-03-21T08:56:20.038147+0000","last_fullsized":"2024-03-21T08:56:20.038147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:52:01.954850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":72,"num_read_kb":3,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070130+0000","last_change":"2024-03-21T08:55:07.532592+0000","last_active":"2024-03-21T08:56:00.070130+0000","last_peered":"2024-03-21T08:56:00.070130+0000","last_clean":"2024-03-21T08:56:00.070130+0000","last_became_active":"2024-03-21T08:55:07.532210+0000","last_became_peered":"2024-03-21T08:55:07.532210+0000","last_unstale":"2024-03-21T08:56:00.070130+0000","last_undegraded":"2024-03-21T08:56:00.070130+0000","last_fullsized":"2024-03-21T08:56:00.070130+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:43:59.775085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"86'17","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076118+0000","last_change":"2024-03-21T08:55:06.229096+0000","last_active":"2024-03-21T08:56:00.076118+0000","last_peered":"2024-03-21T08:56:00.076118+0000","last_clean":"2024-03-21T08:56:00.076118+0000","last_became_active":"2024-03-21T08:55:06.228772+0000","last_became_peered":"2024-03-21T08:55:06.228772+0000","last_unstale":"2024-03-21T08:56:00.076118+0000","last_undegraded":"2024-03-21T08:56:00.076118+0000","last_fullsized":"2024-03-21T08:56:00.076118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:05:31.017977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074681+0000","last_change":"2024-03-21T08:55:07.527968+0000","last_active":"2024-03-21T08:56:00.074681+0000","last_peered":"2024-03-21T08:56:00.074681+0000","last_clean":"2024-03-21T08:56:00.074681+0000","last_became_active":"2024-03-21T08:55:07.523326+0000","last_became_peered":"2024-03-21T08:55:07.523326+0000","last_unstale":"2024-03-21T08:56:00.074681+0000","last_undegraded":"2024-03-21T08:56:00.074681+0000","last_fullsized":"2024-03-21T08:56:00.074681+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:01:22.903576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"86'2","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.082401+0000","last_change":"2024-03-21T08:56:00.081955+0000","last_active":"2024-03-21T08:56:00.082401+0000","last_peered":"2024-03-21T08:56:00.082401+0000","last_clean":"2024-03-21T08:56:00.082401+0000","last_became_active":"2024-03-21T08:56:00.077347+0000","last_became_peered":"2024-03-21T08:56:00.077347+0000","last_unstale":"2024-03-21T08:56:00.082401+0000","last_undegraded":"2024-03-21T08:56:00.082401+0000","last_fullsized":"2024-03-21T08:56:00.082401+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:11:33.812992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072363+0000","last_change":"2024-03-21T08:55:07.532577+0000","last_active":"2024-03-21T08:56:00.072363+0000","last_peered":"2024-03-21T08:56:00.072363+0000","last_clean":"2024-03-21T08:56:00.072363+0000","last_became_active":"2024-03-21T08:55:07.529015+0000","last_became_peered":"2024-03-21T08:55:07.529015+0000","last_unstale":"2024-03-21T08:56:00.072363+0000","last_undegraded":"2024-03-21T08:56:00.072363+0000","last_fullsized":"2024-03-21T08:56:00.072363+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:24.045732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"86'1","reported_seq":20,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069896+0000","last_change":"2024-03-21T08:55:06.238430+0000","last_active":"2024-03-21T08:56:00.069896+0000","last_peered":"2024-03-21T08:56:00.069896+0000","last_clean":"2024-03-21T08:56:00.069896+0000","last_became_active":"2024-03-21T08:55:06.238128+0000","last_became_peered":"2024-03-21T08:55:06.238128+0000","last_unstale":"2024-03-21T08:56:00.069896+0000","last_undegraded":"2024-03-21T08:56:00.069896+0000","last_fullsized":"2024-03-21T08:56:00.069896+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:39:49.839546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066767+0000","last_change":"2024-03-21T08:55:07.540750+0000","last_active":"2024-03-21T08:56:00.066767+0000","last_peered":"2024-03-21T08:56:00.066767+0000","last_clean":"2024-03-21T08:56:00.066767+0000","last_became_active":"2024-03-21T08:55:07.540358+0000","last_became_peered":"2024-03-21T08:55:07.540358+0000","last_unstale":"2024-03-21T08:56:00.066767+0000","last_undegraded":"2024-03-21T08:56:00.066767+0000","last_fullsized":"2024-03-21T08:56:00.066767+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:14:37.590268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:21.039098+0000","last_change":"2024-03-21T08:55:06.236387+0000","last_active":"2024-03-21T08:56:21.039098+0000","last_peered":"2024-03-21T08:56:21.039098+0000","last_clean":"2024-03-21T08:56:21.039098+0000","last_became_active":"2024-03-21T08:55:06.234603+0000","last_became_peered":"2024-03-21T08:55:06.234603+0000","last_unstale":"2024-03-21T08:56:21.039098+0000","last_undegraded":"2024-03-21T08:56:21.039098+0000","last_fullsized":"2024-03-21T08:56:21.039098+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:10:28.921679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":71,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.066553+0000","last_change":"2024-03-21T08:55:07.539011+0000","last_active":"2024-03-21T08:56:00.066553+0000","last_peered":"2024-03-21T08:56:00.066553+0000","last_clean":"2024-03-21T08:56:00.066553+0000","last_became_active":"2024-03-21T08:55:07.538694+0000","last_became_peered":"2024-03-21T08:55:07.538694+0000","last_unstale":"2024-03-21T08:56:00.066553+0000","last_undegraded":"2024-03-21T08:56:00.066553+0000","last_fullsized":"2024-03-21T08:56:00.066553+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T19:58:52.746409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074466+0000","last_change":"2024-03-21T08:55:07.531605+0000","last_active":"2024-03-21T08:56:00.074466+0000","last_peered":"2024-03-21T08:56:00.074466+0000","last_clean":"2024-03-21T08:56:00.074466+0000","last_became_active":"2024-03-21T08:55:07.530207+0000","last_became_peered":"2024-03-21T08:55:07.530207+0000","last_unstale":"2024-03-21T08:56:00.074466+0000","last_undegraded":"2024-03-21T08:56:00.074466+0000","last_fullsized":"2024-03-21T08:56:00.074466+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:29:58.805184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:21.038634+0000","last_change":"2024-03-21T08:55:06.228108+0000","last_active":"2024-03-21T08:56:21.038634+0000","last_peered":"2024-03-21T08:56:21.038634+0000","last_clean":"2024-03-21T08:56:21.038634+0000","last_became_active":"2024-03-21T08:55:06.227281+0000","last_became_peered":"2024-03-21T08:55:06.227281+0000","last_unstale":"2024-03-21T08:56:21.038634+0000","last_undegraded":"2024-03-21T08:56:21.038634+0000","last_fullsized":"2024-03-21T08:56:21.038634+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:24:02.026031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":71,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392869+0000","last_change":"2024-03-21T08:55:07.531090+0000","last_active":"2024-03-21T08:56:00.392869+0000","last_peered":"2024-03-21T08:56:00.392869+0000","last_clean":"2024-03-21T08:56:00.392869+0000","last_became_active":"2024-03-21T08:55:07.530769+0000","last_became_peered":"2024-03-21T08:55:07.530769+0000","last_unstale":"2024-03-21T08:56:00.392869+0000","last_undegraded":"2024-03-21T08:56:00.392869+0000","last_fullsized":"2024-03-21T08:56:00.392869+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:49:51.950743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"86'6","reported_seq":28,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074276+0000","last_change":"2024-03-21T08:55:06.228583+0000","last_active":"2024-03-21T08:56:00.074276+0000","last_peered":"2024-03-21T08:56:00.074276+0000","last_clean":"2024-03-21T08:56:00.074276+0000","last_became_active":"2024-03-21T08:55:06.228222+0000","last_became_peered":"2024-03-21T08:55:06.228222+0000","last_unstale":"2024-03-21T08:56:00.074276+0000","last_undegraded":"2024-03-21T08:56:00.074276+0000","last_fullsized":"2024-03-21T08:56:00.074276+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T09:13:28.254775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070182+0000","last_change":"2024-03-21T08:53:52.956820+0000","last_active":"2024-03-21T08:56:00.070182+0000","last_peered":"2024-03-21T08:56:00.070182+0000","last_clean":"2024-03-21T08:56:00.070182+0000","last_became_active":"2024-03-21T08:53:52.956322+0000","last_became_peered":"2024-03-21T08:53:52.956322+0000","last_unstale":"2024-03-21T08:56:00.070182+0000","last_undegraded":"2024-03-21T08:56:00.070182+0000","last_fullsized":"2024-03-21T08:56:00.070182+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:50:27.622697+0000","last_clean_scrub_stamp":"2024-03-21T08:50:27.622697+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:49:55.824313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068596+0000","last_change":"2024-03-21T08:55:07.529927+0000","last_active":"2024-03-21T08:56:00.068596+0000","last_peered":"2024-03-21T08:56:00.068596+0000","last_clean":"2024-03-21T08:56:00.068596+0000","last_became_active":"2024-03-21T08:55:07.529588+0000","last_became_peered":"2024-03-21T08:55:07.529588+0000","last_unstale":"2024-03-21T08:56:00.068596+0000","last_undegraded":"2024-03-21T08:56:00.068596+0000","last_fullsized":"2024-03-21T08:56:00.068596+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:11:04.122448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"86'4","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070635+0000","last_change":"2024-03-21T08:55:06.236496+0000","last_active":"2024-03-21T08:56:00.070635+0000","last_peered":"2024-03-21T08:56:00.070635+0000","last_clean":"2024-03-21T08:56:00.070635+0000","last_became_active":"2024-03-21T08:55:06.234447+0000","last_became_peered":"2024-03-21T08:55:06.234447+0000","last_unstale":"2024-03-21T08:56:00.070635+0000","last_undegraded":"2024-03-21T08:56:00.070635+0000","last_fullsized":"2024-03-21T08:56:00.070635+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:46:55.322330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076740+0000","last_change":"2024-03-21T08:55:06.232367+0000","last_active":"2024-03-21T08:56:00.076740+0000","last_peered":"2024-03-21T08:56:00.076740+0000","last_clean":"2024-03-21T08:56:00.076740+0000","last_became_active":"2024-03-21T08:55:06.229741+0000","last_became_peered":"2024-03-21T08:55:06.229741+0000","last_unstale":"2024-03-21T08:56:00.076740+0000","last_undegraded":"2024-03-21T08:56:00.076740+0000","last_fullsized":"2024-03-21T08:56:00.076740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:25:51.644926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068871+0000","last_change":"2024-03-21T08:55:07.527366+0000","last_active":"2024-03-21T08:56:00.068871+0000","last_peered":"2024-03-21T08:56:00.068871+0000","last_clean":"2024-03-21T08:56:00.068871+0000","last_became_active":"2024-03-21T08:55:07.527033+0000","last_became_peered":"2024-03-21T08:55:07.527033+0000","last_unstale":"2024-03-21T08:56:00.068871+0000","last_undegraded":"2024-03-21T08:56:00.068871+0000","last_fullsized":"2024-03-21T08:56:00.068871+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:07:38.400820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"86'12","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070147+0000","last_change":"2024-03-21T08:55:06.234078+0000","last_active":"2024-03-21T08:56:00.070147+0000","last_peered":"2024-03-21T08:56:00.070147+0000","last_clean":"2024-03-21T08:56:00.070147+0000","last_became_active":"2024-03-21T08:55:06.233704+0000","last_became_peered":"2024-03-21T08:55:06.233704+0000","last_unstale":"2024-03-21T08:56:00.070147+0000","last_undegraded":"2024-03-21T08:56:00.070147+0000","last_fullsized":"2024-03-21T08:56:00.070147+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:08:00.957142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393156+0000","last_change":"2024-03-21T08:55:07.529668+0000","last_active":"2024-03-21T08:56:00.393156+0000","last_peered":"2024-03-21T08:56:00.393156+0000","last_clean":"2024-03-21T08:56:00.393156+0000","last_became_active":"2024-03-21T08:55:07.528560+0000","last_became_peered":"2024-03-21T08:55:07.528560+0000","last_unstale":"2024-03-21T08:56:00.393156+0000","last_undegraded":"2024-03-21T08:56:00.393156+0000","last_fullsized":"2024-03-21T08:56:00.393156+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T13:16:49.671107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"86'7","reported_seq":27,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074611+0000","last_change":"2024-03-21T08:55:06.241169+0000","last_active":"2024-03-21T08:56:00.074611+0000","last_peered":"2024-03-21T08:56:00.074611+0000","last_clean":"2024-03-21T08:56:00.074611+0000","last_became_active":"2024-03-21T08:55:06.240740+0000","last_became_peered":"2024-03-21T08:55:06.240740+0000","last_unstale":"2024-03-21T08:56:00.074611+0000","last_undegraded":"2024-03-21T08:56:00.074611+0000","last_fullsized":"2024-03-21T08:56:00.074611+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T08:56:31.327983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068048+0000","last_change":"2024-03-21T08:55:07.544243+0000","last_active":"2024-03-21T08:56:00.068048+0000","last_peered":"2024-03-21T08:56:00.068048+0000","last_clean":"2024-03-21T08:56:00.068048+0000","last_became_active":"2024-03-21T08:55:07.543827+0000","last_became_peered":"2024-03-21T08:55:07.543827+0000","last_unstale":"2024-03-21T08:56:00.068048+0000","last_undegraded":"2024-03-21T08:56:00.068048+0000","last_fullsized":"2024-03-21T08:56:00.068048+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:15:32.952933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"86'2","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.078546+0000","last_change":"2024-03-21T08:56:00.078546+0000","last_active":"2024-03-21T08:56:00.078546+0000","last_peered":"2024-03-21T08:56:00.078546+0000","last_clean":"2024-03-21T08:56:00.078546+0000","last_became_active":"2024-03-21T08:56:00.073617+0000","last_became_peered":"2024-03-21T08:56:00.073617+0000","last_unstale":"2024-03-21T08:56:00.078546+0000","last_undegraded":"2024-03-21T08:56:00.078546+0000","last_fullsized":"2024-03-21T08:56:00.078546+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:26:55.511382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,11,0],"acting":[1,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076027+0000","last_change":"2024-03-21T08:55:07.540974+0000","last_active":"2024-03-21T08:56:00.076027+0000","last_peered":"2024-03-21T08:56:00.076027+0000","last_clean":"2024-03-21T08:56:00.076027+0000","last_became_active":"2024-03-21T08:55:07.540246+0000","last_became_peered":"2024-03-21T08:55:07.540246+0000","last_unstale":"2024-03-21T08:56:00.076027+0000","last_undegraded":"2024-03-21T08:56:00.076027+0000","last_fullsized":"2024-03-21T08:56:00.076027+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:14:27.255613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"86'7","reported_seq":26,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392795+0000","last_change":"2024-03-21T08:55:06.231544+0000","last_active":"2024-03-21T08:56:00.392795+0000","last_peered":"2024-03-21T08:56:00.392795+0000","last_clean":"2024-03-21T08:56:00.392795+0000","last_became_active":"2024-03-21T08:55:06.231238+0000","last_became_peered":"2024-03-21T08:55:06.231238+0000","last_unstale":"2024-03-21T08:56:00.392795+0000","last_undegraded":"2024-03-21T08:56:00.392795+0000","last_fullsized":"2024-03-21T08:56:00.392795+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:46:34.831596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072473+0000","last_change":"2024-03-21T08:55:07.541189+0000","last_active":"2024-03-21T08:56:00.072473+0000","last_peered":"2024-03-21T08:56:00.072473+0000","last_clean":"2024-03-21T08:56:00.072473+0000","last_became_active":"2024-03-21T08:55:07.540819+0000","last_became_peered":"2024-03-21T08:55:07.540819+0000","last_unstale":"2024-03-21T08:56:00.072473+0000","last_undegraded":"2024-03-21T08:56:00.072473+0000","last_fullsized":"2024-03-21T08:56:00.072473+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:21:43.879895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"86'3","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071366+0000","last_change":"2024-03-21T08:55:06.231164+0000","last_active":"2024-03-21T08:56:00.071366+0000","last_peered":"2024-03-21T08:56:00.071366+0000","last_clean":"2024-03-21T08:56:00.071366+0000","last_became_active":"2024-03-21T08:55:06.230782+0000","last_became_peered":"2024-03-21T08:55:06.230782+0000","last_unstale":"2024-03-21T08:56:00.071366+0000","last_undegraded":"2024-03-21T08:56:00.071366+0000","last_fullsized":"2024-03-21T08:56:00.071366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:26:56.744957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392895+0000","last_change":"2024-03-21T08:55:07.544379+0000","last_active":"2024-03-21T08:56:00.392895+0000","last_peered":"2024-03-21T08:56:00.392895+0000","last_clean":"2024-03-21T08:56:00.392895+0000","last_became_active":"2024-03-21T08:55:07.541643+0000","last_became_peered":"2024-03-21T08:55:07.541643+0000","last_unstale":"2024-03-21T08:56:00.392895+0000","last_undegraded":"2024-03-21T08:56:00.392895+0000","last_fullsized":"2024-03-21T08:56:00.392895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:54:19.814470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"86'4","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076493+0000","last_change":"2024-03-21T08:55:06.229859+0000","last_active":"2024-03-21T08:56:00.076493+0000","last_peered":"2024-03-21T08:56:00.076493+0000","last_clean":"2024-03-21T08:56:00.076493+0000","last_became_active":"2024-03-21T08:55:06.229532+0000","last_became_peered":"2024-03-21T08:55:06.229532+0000","last_unstale":"2024-03-21T08:56:00.076493+0000","last_undegraded":"2024-03-21T08:56:00.076493+0000","last_fullsized":"2024-03-21T08:56:00.076493+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:57:39.615417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071000+0000","last_change":"2024-03-21T08:55:07.530619+0000","last_active":"2024-03-21T08:56:00.071000+0000","last_peered":"2024-03-21T08:56:00.071000+0000","last_clean":"2024-03-21T08:56:00.071000+0000","last_became_active":"2024-03-21T08:55:07.530293+0000","last_became_peered":"2024-03-21T08:55:07.530293+0000","last_unstale":"2024-03-21T08:56:00.071000+0000","last_undegraded":"2024-03-21T08:56:00.071000+0000","last_fullsized":"2024-03-21T08:56:00.071000+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T17:44:56.242621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"86'5","reported_seq":95,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:20.037738+0000","last_change":"2024-03-21T08:55:06.241581+0000","last_active":"2024-03-21T08:56:20.037738+0000","last_peered":"2024-03-21T08:56:20.037738+0000","last_clean":"2024-03-21T08:56:20.037738+0000","last_became_active":"2024-03-21T08:55:06.241162+0000","last_became_peered":"2024-03-21T08:55:06.241162+0000","last_unstale":"2024-03-21T08:56:20.037738+0000","last_undegraded":"2024-03-21T08:56:20.037738+0000","last_fullsized":"2024-03-21T08:56:20.037738+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:25:15.231840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1170,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070496+0000","last_change":"2024-03-21T08:55:07.532424+0000","last_active":"2024-03-21T08:56:00.070496+0000","last_peered":"2024-03-21T08:56:00.070496+0000","last_clean":"2024-03-21T08:56:00.070496+0000","last_became_active":"2024-03-21T08:55:07.532062+0000","last_became_peered":"2024-03-21T08:55:07.532062+0000","last_unstale":"2024-03-21T08:56:00.070496+0000","last_undegraded":"2024-03-21T08:56:00.070496+0000","last_fullsized":"2024-03-21T08:56:00.070496+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:44:07.970387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"86'3","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074454+0000","last_change":"2024-03-21T08:55:06.232191+0000","last_active":"2024-03-21T08:56:00.074454+0000","last_peered":"2024-03-21T08:56:00.074454+0000","last_clean":"2024-03-21T08:56:00.074454+0000","last_became_active":"2024-03-21T08:55:06.231880+0000","last_became_peered":"2024-03-21T08:55:06.231880+0000","last_unstale":"2024-03-21T08:56:00.074454+0000","last_undegraded":"2024-03-21T08:56:00.074454+0000","last_fullsized":"2024-03-21T08:56:00.074454+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:02:35.434959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"86'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070331+0000","last_change":"2024-03-21T08:55:07.538588+0000","last_active":"2024-03-21T08:56:00.070331+0000","last_peered":"2024-03-21T08:56:00.070331+0000","last_clean":"2024-03-21T08:56:00.070331+0000","last_became_active":"2024-03-21T08:55:07.538268+0000","last_became_peered":"2024-03-21T08:55:07.538268+0000","last_unstale":"2024-03-21T08:56:00.070331+0000","last_undegraded":"2024-03-21T08:56:00.070331+0000","last_fullsized":"2024-03-21T08:56:00.070331+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:41:21.591094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"86'5","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068839+0000","last_change":"2024-03-21T08:55:06.228166+0000","last_active":"2024-03-21T08:56:00.068839+0000","last_peered":"2024-03-21T08:56:00.068839+0000","last_clean":"2024-03-21T08:56:00.068839+0000","last_became_active":"2024-03-21T08:55:06.227852+0000","last_became_peered":"2024-03-21T08:55:06.227852+0000","last_unstale":"2024-03-21T08:56:00.068839+0000","last_undegraded":"2024-03-21T08:56:00.068839+0000","last_fullsized":"2024-03-21T08:56:00.068839+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:05.211271+0000","last_clean_scrub_stamp":"2024-03-21T08:55:05.211271+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T14:56:12.471809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.070562+0000","last_change":"2024-03-21T08:55:07.536854+0000","last_active":"2024-03-21T08:56:00.070562+0000","last_peered":"2024-03-21T08:56:00.070562+0000","last_clean":"2024-03-21T08:56:00.070562+0000","last_became_active":"2024-03-21T08:55:07.536241+0000","last_became_peered":"2024-03-21T08:55:07.536241+0000","last_unstale":"2024-03-21T08:56:00.070562+0000","last_undegraded":"2024-03-21T08:56:00.070562+0000","last_fullsized":"2024-03-21T08:56:00.070562+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:04:21.060162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072594+0000","last_change":"2024-03-21T08:55:07.528084+0000","last_active":"2024-03-21T08:56:00.072594+0000","last_peered":"2024-03-21T08:56:00.072594+0000","last_clean":"2024-03-21T08:56:00.072594+0000","last_became_active":"2024-03-21T08:55:07.527675+0000","last_became_peered":"2024-03-21T08:55:07.527675+0000","last_unstale":"2024-03-21T08:56:00.072594+0000","last_undegraded":"2024-03-21T08:56:00.072594+0000","last_fullsized":"2024-03-21T08:56:00.072594+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:47:21.646078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071316+0000","last_change":"2024-03-21T08:55:07.542384+0000","last_active":"2024-03-21T08:56:00.071316+0000","last_peered":"2024-03-21T08:56:00.071316+0000","last_clean":"2024-03-21T08:56:00.071316+0000","last_became_active":"2024-03-21T08:55:07.541956+0000","last_became_peered":"2024-03-21T08:55:07.541956+0000","last_unstale":"2024-03-21T08:56:00.071316+0000","last_undegraded":"2024-03-21T08:56:00.071316+0000","last_fullsized":"2024-03-21T08:56:00.071316+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T15:54:47.093433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071248+0000","last_change":"2024-03-21T08:55:07.536226+0000","last_active":"2024-03-21T08:56:00.071248+0000","last_peered":"2024-03-21T08:56:00.071248+0000","last_clean":"2024-03-21T08:56:00.071248+0000","last_became_active":"2024-03-21T08:55:07.535711+0000","last_became_peered":"2024-03-21T08:55:07.535711+0000","last_unstale":"2024-03-21T08:56:00.071248+0000","last_undegraded":"2024-03-21T08:56:00.071248+0000","last_fullsized":"2024-03-21T08:56:00.071248+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T20:37:29.252051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068051+0000","last_change":"2024-03-21T08:55:07.536440+0000","last_active":"2024-03-21T08:56:00.068051+0000","last_peered":"2024-03-21T08:56:00.068051+0000","last_clean":"2024-03-21T08:56:00.068051+0000","last_became_active":"2024-03-21T08:55:07.536076+0000","last_became_peered":"2024-03-21T08:55:07.536076+0000","last_unstale":"2024-03-21T08:56:00.068051+0000","last_undegraded":"2024-03-21T08:56:00.068051+0000","last_fullsized":"2024-03-21T08:56:00.068051+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:26:30.750408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.068879+0000","last_change":"2024-03-21T08:55:07.542634+0000","last_active":"2024-03-21T08:56:00.068879+0000","last_peered":"2024-03-21T08:56:00.068879+0000","last_clean":"2024-03-21T08:56:00.068879+0000","last_became_active":"2024-03-21T08:55:07.542317+0000","last_became_peered":"2024-03-21T08:55:07.542317+0000","last_unstale":"2024-03-21T08:56:00.068879+0000","last_undegraded":"2024-03-21T08:56:00.068879+0000","last_fullsized":"2024-03-21T08:56:00.068879+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:18:31.392606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.072408+0000","last_change":"2024-03-21T08:55:07.537660+0000","last_active":"2024-03-21T08:56:00.072408+0000","last_peered":"2024-03-21T08:56:00.072408+0000","last_clean":"2024-03-21T08:56:00.072408+0000","last_became_active":"2024-03-21T08:55:07.537205+0000","last_became_peered":"2024-03-21T08:55:07.537205+0000","last_unstale":"2024-03-21T08:56:00.072408+0000","last_undegraded":"2024-03-21T08:56:00.072408+0000","last_fullsized":"2024-03-21T08:56:00.072408+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T12:39:59.336323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.071576+0000","last_change":"2024-03-21T08:55:07.530140+0000","last_active":"2024-03-21T08:56:00.071576+0000","last_peered":"2024-03-21T08:56:00.071576+0000","last_clean":"2024-03-21T08:56:00.071576+0000","last_became_active":"2024-03-21T08:55:07.529373+0000","last_became_peered":"2024-03-21T08:55:07.529373+0000","last_unstale":"2024-03-21T08:56:00.071576+0000","last_undegraded":"2024-03-21T08:56:00.071576+0000","last_fullsized":"2024-03-21T08:56:00.071576+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T11:10:37.939733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.393067+0000","last_change":"2024-03-21T08:55:07.529218+0000","last_active":"2024-03-21T08:56:00.393067+0000","last_peered":"2024-03-21T08:56:00.393067+0000","last_clean":"2024-03-21T08:56:00.393067+0000","last_became_active":"2024-03-21T08:55:07.528898+0000","last_became_peered":"2024-03-21T08:55:07.528898+0000","last_unstale":"2024-03-21T08:56:00.393067+0000","last_undegraded":"2024-03-21T08:56:00.393067+0000","last_fullsized":"2024-03-21T08:56:00.393067+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:36:49.392688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.074560+0000","last_change":"2024-03-21T08:55:07.531920+0000","last_active":"2024-03-21T08:56:00.074560+0000","last_peered":"2024-03-21T08:56:00.074560+0000","last_clean":"2024-03-21T08:56:00.074560+0000","last_became_active":"2024-03-21T08:55:07.531266+0000","last_became_peered":"2024-03-21T08:55:07.531266+0000","last_unstale":"2024-03-21T08:56:00.074560+0000","last_undegraded":"2024-03-21T08:56:00.074560+0000","last_fullsized":"2024-03-21T08:56:00.074560+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T10:41:19.838525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.392678+0000","last_change":"2024-03-21T08:55:07.530658+0000","last_active":"2024-03-21T08:56:00.392678+0000","last_peered":"2024-03-21T08:56:00.392678+0000","last_clean":"2024-03-21T08:56:00.392678+0000","last_became_active":"2024-03-21T08:55:07.529933+0000","last_became_peered":"2024-03-21T08:55:07.529933+0000","last_unstale":"2024-03-21T08:56:00.392678+0000","last_undegraded":"2024-03-21T08:56:00.392678+0000","last_fullsized":"2024-03-21T08:56:00.392678+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:52:56.698603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069429+0000","last_change":"2024-03-21T08:55:07.529151+0000","last_active":"2024-03-21T08:56:00.069429+0000","last_peered":"2024-03-21T08:56:00.069429+0000","last_clean":"2024-03-21T08:56:00.069429+0000","last_became_active":"2024-03-21T08:55:07.528511+0000","last_became_peered":"2024-03-21T08:55:07.528511+0000","last_unstale":"2024-03-21T08:56:00.069429+0000","last_undegraded":"2024-03-21T08:56:00.069429+0000","last_fullsized":"2024-03-21T08:56:00.069429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T16:40:27.720391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.069226+0000","last_change":"2024-03-21T08:55:07.541519+0000","last_active":"2024-03-21T08:56:00.069226+0000","last_peered":"2024-03-21T08:56:00.069226+0000","last_clean":"2024-03-21T08:56:00.069226+0000","last_became_active":"2024-03-21T08:55:07.541171+0000","last_became_peered":"2024-03-21T08:55:07.541171+0000","last_unstale":"2024-03-21T08:56:00.069226+0000","last_undegraded":"2024-03-21T08:56:00.069226+0000","last_fullsized":"2024-03-21T08:56:00.069226+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:10:36.249778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-03-21T08:56:00.076625+0000","last_change":"2024-03-21T08:56:00.076625+0000","last_active":"2024-03-21T08:56:00.076625+0000","last_peered":"2024-03-21T08:56:00.076625+0000","last_clean":"2024-03-21T08:56:00.076625+0000","last_became_active":"2024-03-21T08:56:00.076266+0000","last_became_peered":"2024-03-21T08:56:00.076266+0000","last_unstale":"2024-03-21T08:56:00.076625+0000","last_undegraded":"2024-03-21T08:56:00.076625+0000","last_fullsized":"2024-03-21T08:56:00.076625+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-21T08:55:06.213105+0000","last_clean_scrub_stamp":"2024-03-21T08:55:06.213105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-22T18:01:55.701867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":280365,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67859,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":757,"num_read_kb":457,"num_write":236,"num_write_kb":132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":552960,"data_stored":203577,"data_compressed":5550,"data_compressed_allocated":12288,"data_compressed_original":24576,"omap_allocated":0,"internal_metadata":0},"log_size":205,"ondisk_log_size":205,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":81,"seq":347892351005,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":944,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":966656,"data_stored":461228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":73,"seq":313532612642,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28108,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717044,"statfs":{"total":95995035648,"available":95966253056,"internally_reserved":0,"allocated":1114112,"data_stored":1113672,"data_compressed":37511,"data_compressed_allocated":49152,"data_compressed_original":598016,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":68,"seq":292057776168,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":987136,"data_stored":461136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,5,6,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":60,"seq":257698037804,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":956,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":978944,"data_stored":461492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":52,"seq":223338299442,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":1054032,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462967,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":460853,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":522812,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738453,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":462192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411403,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":523032,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182470,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":987136,"data_stored":460678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920833,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":461105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724604,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1048576,"data_stored":1053319,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34020,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35661,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":62798,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":63018,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":2820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3533,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":63173,"data_compressed":1850,"data_compressed_allocated":4096,"data_compressed_original":8192,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-21T08:56:28.003 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-21T08:56:28.003 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-21T08:56:28.003 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-21T08:56:28.004 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph health --format=json 2024-03-21T08:56:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:27 smithi105 ceph-mon[35993]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:28.246 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:27 smithi100 ceph-mon[36306]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:28.367 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:27 smithi087 ceph-mon[32385]: pgmap v332: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:29.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:28 smithi105 ceph-mon[35993]: from='client.24773 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:56:29.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:28 smithi100 ceph-mon[36306]: from='client.24773 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:56:29.396 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:28 smithi087 ceph-mon[32385]: from='client.24773 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-21T08:56:29.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 -- 172.21.15.87:0/1289041217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 msgr2=0x7f3984106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:29.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 --2- 172.21.15.87:0/1289041217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f3984106dd0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f39780099c0 tx=0x7f3978009cd0 comp rx=0 tx=0).stop 2024-03-21T08:56:29.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 -- 172.21.15.87:0/1289041217 shutdown_connections 2024-03-21T08:56:29.950 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 --2- 172.21.15.87:0/1289041217 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39841023d0 0x7f39841027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 --2- 172.21.15.87:0/1289041217 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841074b0 0x7f398410f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 --2- 172.21.15.87:0/1289041217 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f3984106dd0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 -- 172.21.15.87:0/1289041217 >> 172.21.15.87:0/1289041217 conn(0x7f39840fdc40 msgr2=0x7f3984100060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 -- 172.21.15.87:0/1289041217 shutdown_connections 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.952+0000 7f3989637700 1 -- 172.21.15.87:0/1289041217 wait complete. 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 Processor -- start 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 -- start start 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 0x7f3984074490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f39840749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39841074b0 0x7f3984074f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3984112490 con 0x7f3984102d80 2024-03-21T08:56:29.951 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3984112310 con 0x7f39841074b0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3989637700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3984112610 con 0x7f39841023d0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3982ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 0x7f3984074490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f39827fc700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f39840749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3982ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 0x7f3984074490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58406/0 (socket says 172.21.15.87:58406) 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f3982ffd700 1 -- 172.21.15.87:0/3398934251 learned_addr learned my addr 172.21.15.87:0/3398934251 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.953+0000 7f39837fe700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39841074b0 0x7f3984074f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3982ffd700 1 -- 172.21.15.87:0/3398934251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39841074b0 msgr2=0x7f3984074f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3982ffd700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39841074b0 0x7f3984074f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3982ffd700 1 -- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 msgr2=0x7f39840749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3982ffd700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f39840749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3982ffd700 1 -- 172.21.15.87:0/3398934251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39841a6e90 con 0x7f39841023d0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f39827fc700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f39840749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3982ffd700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 0x7f3984074490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f397400e9a0 tx=0x7f397400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:29.952 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397400ca10 con 0x7f39841023d0 2024-03-21T08:56:29.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39841a7180 con 0x7f39841023d0 2024-03-21T08:56:29.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f397400cbb0 con 0x7f39841023d0 2024-03-21T08:56:29.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974018680 con 0x7f39841023d0 2024-03-21T08:56:29.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.954+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39841a7690 con 0x7f39841023d0 2024-03-21T08:56:29.953 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.955+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3984068e50 con 0x7f39841023d0 2024-03-21T08:56:29.953 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:29 smithi087 ceph-mon[32385]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:29.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.956+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3974010430 con 0x7f39841023d0 2024-03-21T08:56:29.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.957+0000 7f396bfff700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f396c0a2fb0 0x7f396c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:29.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.957+0000 7f39827fc700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f396c0a2fb0 0x7f396c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:29.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.957+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f39740cf9c0 con 0x7f39841023d0 2024-03-21T08:56:29.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.957+0000 7f39827fc700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f396c0a2fb0 0x7f396c0a5470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3978000c00 tx=0x7f3978005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:29.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:29.960+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f397409c7c0 con 0x7f39841023d0 2024-03-21T08:56:30.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.202+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f398402da30 con 0x7f39841023d0 2024-03-21T08:56:30.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.202+0000 7f396bfff700 1 -- 172.21.15.87:0/3398934251 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f397409c0c0 con 0x7f39841023d0 2024-03-21T08:56:30.197 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:30.197 INFO:teuthology.orchestra.run.smithi087.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-21T08:56:30.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f396c0a2fb0 msgr2=0x7f396c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:30.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f396c0a2fb0 0x7f396c0a5470 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3978000c00 tx=0x7f3978005fb0 comp rx=0 tx=0).stop 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 msgr2=0x7f3984074490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 0x7f3984074490 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f397400e9a0 tx=0x7f397400ecb0 comp rx=0 tx=0).stop 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 shutdown_connections 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f396c0a2fb0 0x7f396c0a5470 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f39841074b0 0x7f3984074f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f39841023d0 0x7f3984074490 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 --2- 172.21.15.87:0/3398934251 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3984102d80 0x7f39840749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.204+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 >> 172.21.15.87:0/3398934251 conn(0x7f39840fdc40 msgr2=0x7f39840fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.205+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 shutdown_connections 2024-03-21T08:56:30.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:30.205+0000 7f3989637700 1 -- 172.21.15.87:0/3398934251 wait complete. 2024-03-21T08:56:30.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:29 smithi105 ceph-mon[35993]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:30.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:29 smithi100 ceph-mon[36306]: pgmap v333: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:30.903 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-21T08:56:30.904 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:56:31.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 -- 172.21.15.87:0/2561685200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 msgr2=0x7f788c102d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 --2- 172.21.15.87:0/2561685200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 0x7f788c102d30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7880009900 tx=0x7f788002bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:31.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 -- 172.21.15.87:0/2561685200 shutdown_connections 2024-03-21T08:56:31.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 --2- 172.21.15.87:0/2561685200 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c103270 0x7f788c109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 --2- 172.21.15.87:0/2561685200 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 0x7f788c102d30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 --2- 172.21.15.87:0/2561685200 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c1016c0 0x7f788c101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 -- 172.21.15.87:0/2561685200 >> 172.21.15.87:0/2561685200 conn(0x7f788c0fcc50 msgr2=0x7f788c0ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 -- 172.21.15.87:0/2561685200 shutdown_connections 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.018+0000 7f7891e59640 1 -- 172.21.15.87:0/2561685200 wait complete. 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 Processor -- start 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 -- start start 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 0x7f788c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 0x7f788c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c103270 0x7f788c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c10d390 con 0x7f788c1016c0 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c10d210 con 0x7f788c103270 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f7891e59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c10d510 con 0x7f788c1028b0 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 0x7f788c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.016 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 0x7f788c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34674/0 (socket says 172.21.15.87:34674) 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 -- 172.21.15.87:0/2951965818 learned_addr learned my addr 172.21.15.87:0/2951965818 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788affd640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 0x7f788c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788bfff640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c103270 0x7f788c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 -- 172.21.15.87:0/2951965818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 msgr2=0x7f788c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 0x7f788c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 -- 172.21.15.87:0/2951965818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c103270 msgr2=0x7f788c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c103270 0x7f788c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 -- 172.21.15.87:0/2951965818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c19f5f0 con 0x7f788c1016c0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.019+0000 7f788b7fe640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 0x7f788c1979d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9c0 tx=0x7f787c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.020+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c00cd80 con 0x7f788c1016c0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.020+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c19f8e0 con 0x7f788c1016c0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.020+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f787c0035f0 con 0x7f788c1016c0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.020+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c010690 con 0x7f788c1016c0 2024-03-21T08:56:31.017 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.020+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c19fdb0 con 0x7f788c1016c0 2024-03-21T08:56:31.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.021+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c1089e0 con 0x7f788c1016c0 2024-03-21T08:56:31.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.022+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f787c0037a0 con 0x7f788c1016c0 2024-03-21T08:56:31.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.022+0000 7f7888ff9640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f78740a2f20 0x7f78740a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.022+0000 7f788affd640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f78740a2f20 0x7f78740a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.022+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f787c0cf720 con 0x7f788c1016c0 2024-03-21T08:56:31.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.023+0000 7f788affd640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f78740a2f20 0x7f78740a53e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f788002c240 tx=0x7f788002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.022 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.027+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f787c09c580 con 0x7f788c1016c0 2024-03-21T08:56:31.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:30 smithi087 ceph-mon[32385]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-21T08:56:31.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:30 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3398934251' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T08:56:31.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.203+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f788c0622a0 con 0x7f788c1016c0 2024-03-21T08:56:31.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.204+0000 7f7888ff9640 1 -- 172.21.15.87:0/2951965818 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7f787c09be80 con 0x7f788c1016c0 2024-03-21T08:56:31.199 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:31.200 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:56:31.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f78740a2f20 msgr2=0x7f78740a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f78740a2f20 0x7f78740a53e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f788002c240 tx=0x7f788002df90 comp rx=0 tx=0).stop 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 msgr2=0x7f788c1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 0x7f788c1979d0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f787c00e9c0 tx=0x7f787c00ee80 comp rx=0 tx=0).stop 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 shutdown_connections 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f78740a2f20 0x7f78740a53e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f788c103270 0x7f788c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f788c1028b0 0x7f788c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 --2- 172.21.15.87:0/2951965818 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f788c1016c0 0x7f788c1979d0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 >> 172.21.15.87:0/2951965818 conn(0x7f788c0fcc50 msgr2=0x7f788c105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.206+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 shutdown_connections 2024-03-21T08:56:31.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.207+0000 7f7891e59640 1 -- 172.21.15.87:0/2951965818 wait complete. 2024-03-21T08:56:31.213 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:55:04.486296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:55:05.756309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:56:31.213 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:56:31.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:30 smithi105 ceph-mon[35993]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-21T08:56:31.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:30 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3398934251' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T08:56:31.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:30 smithi100 ceph-mon[36306]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-03-21T08:56:31.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:30 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3398934251' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-21T08:56:31.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 -- 172.21.15.87:0/2575039256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102170 msgr2=0x7fc84c108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.319 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2575039256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102170 0x7fc84c108a00 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fc844009900 tx=0x7fc84402bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 -- 172.21.15.87:0/2575039256 shutdown_connections 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2575039256 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c102170 0x7fc84c108a00 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2575039256 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc84c1017b0 0x7fc84c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2575039256 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c1005c0 0x7fc84c1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 -- 172.21.15.87:0/2575039256 >> 172.21.15.87:0/2575039256 conn(0x7fc84c0fbb50 msgr2=0x7fc84c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 -- 172.21.15.87:0/2575039256 shutdown_connections 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.322+0000 7fc8534ac640 1 -- 172.21.15.87:0/2575039256 wait complete. 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 Processor -- start 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 -- start start 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 0x7fc84c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc84c1017b0 0x7fc84c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c102170 0x7fc84c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.320 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c10c240 con 0x7fc84c1005c0 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c10c0c0 con 0x7fc84c1017b0 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8534ac640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84c10c3c0 con 0x7fc84c102170 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 0x7fc84c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 0x7fc84c196850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34692/0 (socket says 172.21.15.87:34692) 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 -- 172.21.15.87:0/2397158996 learned_addr learned my addr 172.21.15.87:0/2397158996 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc850a47640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc84c1017b0 0x7fc84c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851a49640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c102170 0x7fc84c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 -- 172.21.15.87:0/2397158996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c102170 msgr2=0x7fc84c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c102170 0x7fc84c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 -- 172.21.15.87:0/2397158996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc84c1017b0 msgr2=0x7fc84c196d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc84c1017b0 0x7fc84c196d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 -- 172.21.15.87:0/2397158996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc84c19e470 con 0x7fc84c1005c0 2024-03-21T08:56:31.321 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851a49640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c102170 0x7fc84c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc851248640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 0x7fc84c196850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc84800eb70 tx=0x7fc84800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.323+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848003b70 con 0x7fc84c1005c0 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.324+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84c19e760 con 0x7fc84c1005c0 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.324+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc84800ee70 con 0x7fc84c1005c0 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.324+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848010830 con 0x7fc84c1005c0 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.324+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc84c19ec60 con 0x7fc84c1005c0 2024-03-21T08:56:31.322 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.324+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84c107960 con 0x7fc84c1005c0 2024-03-21T08:56:31.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.326+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc848004a40 con 0x7fc84c1005c0 2024-03-21T08:56:31.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.326+0000 7fc8427fc640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8380a2f20 0x7fc8380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.326+0000 7fc850a47640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8380a2f20 0x7fc8380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.326+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc8480cfba0 con 0x7fc84c1005c0 2024-03-21T08:56:31.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.326+0000 7fc850a47640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8380a2f20 0x7fc8380a53e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0095f0 tx=0x7fc83c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.330+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc84809ca00 con 0x7fc84c1005c0 2024-03-21T08:56:31.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.503+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc84c0622d0 con 0x7fc84c1005c0 2024-03-21T08:56:31.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.503+0000 7fc8427fc640 1 -- 172.21.15.87:0/2397158996 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc84809c300 con 0x7fc84c1005c0 2024-03-21T08:56:31.498 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:56:31.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8380a2f20 msgr2=0x7fc8380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8380a2f20 0x7fc8380a53e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc83c0095f0 tx=0x7fc83c005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 msgr2=0x7fc84c196850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 0x7fc84c196850 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc84800eb70 tx=0x7fc84800c6a0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 shutdown_connections 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8380a2f20 0x7fc8380a53e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc84c102170 0x7fc84c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc84c1017b0 0x7fc84c196d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 --2- 172.21.15.87:0/2397158996 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc84c1005c0 0x7fc84c196850 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 >> 172.21.15.87:0/2397158996 conn(0x7fc84c0fbb50 msgr2=0x7fc84c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 shutdown_connections 2024-03-21T08:56:31.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.505+0000 7fc8534ac640 1 -- 172.21.15.87:0/2397158996 wait complete. 2024-03-21T08:56:31.511 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:56:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 -- 172.21.15.87:0/2890214997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c102160 msgr2=0x7ffb0c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/2890214997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c102160 0x7ffb0c1089f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7ffb08009900 tx=0x7ffb0802bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 -- 172.21.15.87:0/2890214997 shutdown_connections 2024-03-21T08:56:31.617 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/2890214997 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c102160 0x7ffb0c1089f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/2890214997 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/2890214997 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0c1005b0 0x7ffb0c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 -- 172.21.15.87:0/2890214997 >> 172.21.15.87:0/2890214997 conn(0x7ffb0c0fbb40 msgr2=0x7ffb0c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 -- 172.21.15.87:0/2890214997 shutdown_connections 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.620+0000 7ffb13ddf640 1 -- 172.21.15.87:0/2890214997 wait complete. 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 Processor -- start 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 -- start start 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0c1005b0 0x7ffb0c06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c19c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb0c102160 0x7ffb0c19ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10c230 con 0x7ffb0c1017a0 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10c0b0 con 0x7ffb0c102160 2024-03-21T08:56:31.618 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c10c3b0 con 0x7ffb0c1005b0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c19c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c19c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34714/0 (socket says 172.21.15.87:34714) 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 -- 172.21.15.87:0/950535292 learned_addr learned my addr 172.21.15.87:0/950535292 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb11b7b640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0c1005b0 0x7ffb0c06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1237c640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb0c102160 0x7ffb0c19ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 -- 172.21.15.87:0/950535292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0c1005b0 msgr2=0x7ffb0c06a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0c1005b0 0x7ffb0c06a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 -- 172.21.15.87:0/950535292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb0c102160 msgr2=0x7ffb0c19ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb0c102160 0x7ffb0c19ecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 -- 172.21.15.87:0/950535292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c19f600 con 0x7ffb0c1017a0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb1137a640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c19c6d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00e9f0 tx=0x7ffafc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc00cd80 con 0x7ffb0c1017a0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb0c19f890 con 0x7ffb0c1017a0 2024-03-21T08:56:31.619 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ffafc010910 con 0x7ffb0c1017a0 2024-03-21T08:56:31.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.621+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffafc0188f0 con 0x7ffb0c1017a0 2024-03-21T08:56:31.620 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.622+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb0c19fe30 con 0x7ffb0c1017a0 2024-03-21T08:56:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.623+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb0c1078d0 con 0x7ffb0c1017a0 2024-03-21T08:56:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.623+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ffafc010ac0 con 0x7ffb0c1017a0 2024-03-21T08:56:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.624+0000 7ffb02ffd640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffaf80a2f20 0x7ffaf80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.624+0000 7ffb11b7b640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffaf80a2f20 0x7ffaf80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.624+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ffafc0cf750 con 0x7ffb0c1017a0 2024-03-21T08:56:31.621 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.624+0000 7ffb11b7b640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffaf80a2f20 0x7ffaf80a53e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c101600 tx=0x7ffb04009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.623 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.628+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ffafc09bf60 con 0x7ffb0c1017a0 2024-03-21T08:56:31.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.802+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ffb0c0622a0 con 0x7ffb0c1017a0 2024-03-21T08:56:31.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.802+0000 7ffb02ffd640 1 -- 172.21.15.87:0/950535292 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7ffafc0a0000 con 0x7ffb0c1017a0 2024-03-21T08:56:31.797 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffaf80a2f20 msgr2=0x7ffaf80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffaf80a2f20 0x7ffaf80a53e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c101600 tx=0x7ffb04009500 comp rx=0 tx=0).stop 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 msgr2=0x7ffb0c19c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c19c6d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ffafc00e9f0 tx=0x7ffafc00eeb0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 shutdown_connections 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ffaf80a2f20 0x7ffaf80a53e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ffb0c102160 0x7ffb0c19ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ffb0c1017a0 0x7ffb0c19c6d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 --2- 172.21.15.87:0/950535292 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ffb0c1005b0 0x7ffb0c06a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.804+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 >> 172.21.15.87:0/950535292 conn(0x7ffb0c0fbb40 msgr2=0x7ffb0c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.805+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 shutdown_connections 2024-03-21T08:56:31.800 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.805+0000 7ffb13ddf640 1 -- 172.21.15.87:0/950535292 wait complete. 2024-03-21T08:56:31.811 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:56:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 -- 172.21.15.87:0/2643935155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 msgr2=0x7f46d4106eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 --2- 172.21.15.87:0/2643935155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4106eb0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f46bc009900 tx=0x7f46bc02bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 -- 172.21.15.87:0/2643935155 shutdown_connections 2024-03-21T08:56:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 --2- 172.21.15.87:0/2643935155 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d41073f0 0x7f46d41097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 --2- 172.21.15.87:0/2643935155 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4106eb0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.916 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 --2- 172.21.15.87:0/2643935155 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46d4102190 0x7f46d4104580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 -- 172.21.15.87:0/2643935155 >> 172.21.15.87:0/2643935155 conn(0x7f46d40fbb40 msgr2=0x7f46d40fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 -- 172.21.15.87:0/2643935155 shutdown_connections 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.919+0000 7f46d8fda640 1 -- 172.21.15.87:0/2643935155 wait complete. 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 Processor -- start 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 -- start start 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46d4102190 0x7f46d4101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4100140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d41073f0 0x7f46d4100680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d410d330 con 0x7f46d4104ac0 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d410d1b0 con 0x7f46d4102190 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d8fda640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d410d4b0 con 0x7f46d41073f0 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4100140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4100140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34726/0 (socket says 172.21.15.87:34726) 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 -- 172.21.15.87:0/774663823 learned_addr learned my addr 172.21.15.87:0/774663823 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d259c640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46d4102190 0x7f46d4101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46d2d9d640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d41073f0 0x7f46d4100680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 -- 172.21.15.87:0/774663823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d41073f0 msgr2=0x7f46d4100680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d41073f0 0x7f46d4100680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 -- 172.21.15.87:0/774663823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46d4102190 msgr2=0x7f46d4101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46d4102190 0x7f46d4101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 -- 172.21.15.87:0/774663823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46d4100f10 con 0x7f46d4104ac0 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cbfff640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4100140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.920+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc041070 con 0x7f46d4104ac0 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.921+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f46bc004450 con 0x7f46d4104ac0 2024-03-21T08:56:31.918 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.921+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d41a1ac0 con 0x7f46d4104ac0 2024-03-21T08:56:31.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.921+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc034440 con 0x7f46d4104ac0 2024-03-21T08:56:31.919 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.921+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d41a2010 con 0x7f46d4104ac0 2024-03-21T08:56:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.921+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d40fd630 con 0x7f46d4104ac0 2024-03-21T08:56:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.922+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f46bc044070 con 0x7f46d4104ac0 2024-03-21T08:56:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.923+0000 7f46cb7fe640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f46c00a2f20 0x7f46c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.923+0000 7f46d259c640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f46c00a2f20 0x7f46c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.923+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f46bc0f0e60 con 0x7f46d4104ac0 2024-03-21T08:56:31.920 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.924+0000 7f46d259c640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f46c00a2f20 0x7f46c00a53e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46c40095f0 tx=0x7f46c4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:31.922 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:31.928+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f46bc0bd700 con 0x7f46d4104ac0 2024-03-21T08:56:32.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.105+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f46d40622a0 con 0x7f46d4104ac0 2024-03-21T08:56:32.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.106+0000 7f46cb7fe640 1 -- 172.21.15.87:0/774663823 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f46bc0c17a0 con 0x7f46d4104ac0 2024-03-21T08:56:32.101 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.107+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f46c00a2f20 msgr2=0x7f46c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.107+0000 7f46d8fda640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f46c00a2f20 0x7f46c00a53e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f46c40095f0 tx=0x7f46c4005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.107+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 msgr2=0x7f46d4100140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.107+0000 7f46d8fda640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4100140 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f46bc000c00 tx=0x7f46bc02c640 comp rx=0 tx=0).stop 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 shutdown_connections 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f46c00a2f20 0x7f46c00a53e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f46d41073f0 0x7f46d4100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f46d4104ac0 0x7f46d4100140 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 --2- 172.21.15.87:0/774663823 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f46d4102190 0x7f46d4101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 >> 172.21.15.87:0/774663823 conn(0x7f46d40fbb40 msgr2=0x7f46d4102c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:32.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 shutdown_connections 2024-03-21T08:56:32.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.108+0000 7f46d8fda640 1 -- 172.21.15.87:0/774663823 wait complete. 2024-03-21T08:56:32.113 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:31 smithi087 ceph-mon[32385]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T08:56:32.113 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:31 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2951965818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:32.113 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:31 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2397158996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:32.113 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:31 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/950535292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:32.114 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:56:32.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 -- 172.21.15.87:0/3647666049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 msgr2=0x7fa2e01009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 --2- 172.21.15.87:0/3647666049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e01009e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fa2d0009ca0 tx=0x7fa2d002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 -- 172.21.15.87:0/3647666049 shutdown_connections 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 --2- 172.21.15.87:0/3647666049 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 0x7fa2e0108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 --2- 172.21.15.87:0/3647666049 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 0x7fa2e0101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 --2- 172.21.15.87:0/3647666049 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e01009e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 -- 172.21.15.87:0/3647666049 >> 172.21.15.87:0/3647666049 conn(0x7fa2e00fbb30 msgr2=0x7fa2e00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 -- 172.21.15.87:0/3647666049 shutdown_connections 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.220+0000 7fa2e5587640 1 -- 172.21.15.87:0/3647666049 wait complete. 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 Processor -- start 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 -- start start 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e0069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 0x7fa2e019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 0x7fa2e019ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010c230 con 0x7fa2e01005c0 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010c0b0 con 0x7fa2e01017b0 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2e5587640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e010c3b0 con 0x7fa2e0102170 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2deffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e0069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2deffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e0069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34736/0 (socket says 172.21.15.87:34736) 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2deffd640 1 -- 172.21.15.87:0/1510364376 learned_addr learned my addr 172.21.15.87:0/1510364376 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2df7fe640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 0x7fa2e019ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.221+0000 7fa2de7fc640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 0x7fa2e019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2deffd640 1 -- 172.21.15.87:0/1510364376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 msgr2=0x7fa2e019ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2deffd640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 0x7fa2e019ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2deffd640 1 -- 172.21.15.87:0/1510364376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 msgr2=0x7fa2e019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2deffd640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 0x7fa2e019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2deffd640 1 -- 172.21.15.87:0/1510364376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e019f5d0 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2df7fe640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 0x7fa2e019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2de7fc640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 0x7fa2e019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2deffd640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e0069390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00324c0 tx=0x7fa2d0032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0041020 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa2d0034cd0 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e019f860 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0039040 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.222+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e019fd50 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.223+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e01078e0 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.224+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa2d0044020 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.224+0000 7fa2c7fff640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2c80a2f20 0x7fa2c80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.225+0000 7fa2de7fc640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2c80a2f20 0x7fa2c80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.225+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa2d00f1420 con 0x7fa2e01005c0 2024-03-21T08:56:32.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.225+0000 7fa2de7fc640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2c80a2f20 0x7fa2c80a53e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0069100 tx=0x7fa2d4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:32.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.229+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa2d004b020 con 0x7fa2e01005c0 2024-03-21T08:56:32.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:31 smithi105 ceph-mon[35993]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T08:56:32.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:31 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2951965818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:32.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:31 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2397158996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:32.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:31 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/950535292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:32.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:31 smithi100 ceph-mon[36306]: pgmap v334: 97 pgs: 97 active+clean; 643 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 255 B/s wr, 10 op/s 2024-03-21T08:56:32.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:31 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2951965818' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:32.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:31 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2397158996' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:32.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:31 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/950535292' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:32.437 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.442+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2e00622a0 con 0x7fa2e01005c0 2024-03-21T08:56:32.438 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.443+0000 7fa2c7fff640 1 -- 172.21.15.87:0/1510364376 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11460 (secure 0 0 0) 0x7fa2d00be310 con 0x7fa2e01005c0 2024-03-21T08:56:32.438 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:32.438 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:55:12.955598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24451,24482,24425,24487,24403]},"id":1}]} 2024-03-21T08:56:32.439 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:56:32.440 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2c80a2f20 msgr2=0x7fa2c80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2c80a2f20 0x7fa2c80a53e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0069100 tx=0x7fa2d4005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 msgr2=0x7fa2e0069390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e0069390 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00324c0 tx=0x7fa2d0032f40 comp rx=0 tx=0).stop 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 shutdown_connections 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa2c80a2f20 0x7fa2c80a53e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa2e0102170 0x7fa2e019ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa2e01017b0 0x7fa2e019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 --2- 172.21.15.87:0/1510364376 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa2e01005c0 0x7fa2e0069390 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 >> 172.21.15.87:0/1510364376 conn(0x7fa2e00fbb30 msgr2=0x7fa2e0104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.445+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 shutdown_connections 2024-03-21T08:56:32.441 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.446+0000 7fa2e5587640 1 -- 172.21.15.87:0/1510364376 wait complete. 2024-03-21T08:56:32.452 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:56:32.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- 172.21.15.87:0/616770194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec41016e0 msgr2=0x7fcec4101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- 172.21.15.87:0/616770194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec41016e0 0x7fcec4101b00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fcec0009900 tx=0x7fcec002bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:32.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- 172.21.15.87:0/616770194 shutdown_connections 2024-03-21T08:56:32.556 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- 172.21.15.87:0/616770194 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec4103290 0x7fcec4109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- 172.21.15.87:0/616770194 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec41028d0 0x7fcec4102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- 172.21.15.87:0/616770194 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec41016e0 0x7fcec4101b00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- 172.21.15.87:0/616770194 >> 172.21.15.87:0/616770194 conn(0x7fcec40fcc50 msgr2=0x7fcec40ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- 172.21.15.87:0/616770194 shutdown_connections 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- 172.21.15.87:0/616770194 wait complete. 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 Processor -- start 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- start start 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec41016e0 0x7fcec4197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41028d0 0x7fcec4197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.557 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 0x7fcec419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec410d350 con 0x7fcec4103290 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec410d1d0 con 0x7fcec41028d0 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcecb83d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec410d4d0 con 0x7fcec41016e0 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.559+0000 7fcec9dda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 0x7fcec419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 0x7fcec419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34746/0 (socket says 172.21.15.87:34746) 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec95d9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec41016e0 0x7fcec4197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 -- 172.21.15.87:0/4075241862 learned_addr learned my addr 172.21.15.87:0/4075241862 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec8dd8640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41028d0 0x7fcec4197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 -- 172.21.15.87:0/4075241862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec41016e0 msgr2=0x7fcec4197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.558 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec41016e0 0x7fcec4197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 -- 172.21.15.87:0/4075241862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41028d0 msgr2=0x7fcec4197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41028d0 0x7fcec4197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 -- 172.21.15.87:0/4075241862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec419f670 con 0x7fcec4103290 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcec9dda640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 0x7fcec419f000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00eb10 tx=0x7fcebc00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc004b20 con 0x7fcec4103290 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcebc004cc0 con 0x7fcec4103290 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419f960 con 0x7fcec4103290 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc010760 con 0x7fcec4103290 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.560+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec419feb0 con 0x7fcec4103290 2024-03-21T08:56:32.559 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.561+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec4108ad0 con 0x7fcec4103290 2024-03-21T08:56:32.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.562+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcebc005020 con 0x7fcec4103290 2024-03-21T08:56:32.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.563+0000 7fceba7fc640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fceb00a2fa0 0x7fceb00a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.563+0000 7fcec95d9640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fceb00a2fa0 0x7fceb00a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.563+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcebc0cf8b0 con 0x7fcec4103290 2024-03-21T08:56:32.560 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.563+0000 7fcec95d9640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fceb00a2fa0 0x7fceb00a5460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcec0000c00 tx=0x7fcec0005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:32.562 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.567+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcebc09c710 con 0x7fcec4103290 2024-03-21T08:56:32.739 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.744+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcec40622a0 con 0x7fcec4103290 2024-03-21T08:56:32.740 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.745+0000 7fceba7fc640 1 -- 172.21.15.87:0/4075241862 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7fcebc09c010 con 0x7fcec4103290 2024-03-21T08:56:32.740 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:32.741 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:56:32.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fceb00a2fa0 msgr2=0x7fceb00a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fceb00a2fa0 0x7fceb00a5460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcec0000c00 tx=0x7fcec0005fd0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 msgr2=0x7fcec419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 0x7fcec419f000 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcebc00eb10 tx=0x7fcebc00c2d0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 shutdown_connections 2024-03-21T08:56:32.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fceb00a2fa0 0x7fceb00a5460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fcec4103290 0x7fcec419f000 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.744 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fcec41028d0 0x7fcec4197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 --2- 172.21.15.87:0/4075241862 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fcec41016e0 0x7fcec4197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 >> 172.21.15.87:0/4075241862 conn(0x7fcec40fcc50 msgr2=0x7fcec4105ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:32.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 shutdown_connections 2024-03-21T08:56:32.745 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.748+0000 7fcecb83d640 1 -- 172.21.15.87:0/4075241862 wait complete. 2024-03-21T08:56:32.754 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:55:04.486296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:55:05.756309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:56:32.754 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 -- 172.21.15.87:0/3072163896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6301005a0 msgr2=0x7fb6301009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 --2- 172.21.15.87:0/3072163896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6301005a0 0x7fb6301009c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fb620009ca0 tx=0x7fb62002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 -- 172.21.15.87:0/3072163896 shutdown_connections 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 --2- 172.21.15.87:0/3072163896 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630102150 0x7fb6301089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 --2- 172.21.15.87:0/3072163896 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb630101790 0x7fb630101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 --2- 172.21.15.87:0/3072163896 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb6301005a0 0x7fb6301009c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 -- 172.21.15.87:0/3072163896 >> 172.21.15.87:0/3072163896 conn(0x7fb6300fbb30 msgr2=0x7fb6300fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 -- 172.21.15.87:0/3072163896 shutdown_connections 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.863+0000 7fb634d5b640 1 -- 172.21.15.87:0/3072163896 wait complete. 2024-03-21T08:56:32.861 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 Processor -- start 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 -- start start 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 0x7fb6301044e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630101790 0x7fb630102b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 0x7fb6301030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63010c200 con 0x7fb630102150 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63010c080 con 0x7fb6301005a0 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb634d5b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63010c380 con 0x7fb630101790 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb62e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 0x7fb6301044e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 0x7fb6301030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb62e59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 0x7fb6301044e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:40730/0 (socket says 172.21.15.87:40730) 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb62dd9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630101790 0x7fb630102b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.862 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb62e59c640 1 -- 172.21.15.87:0/4068523835 learned_addr learned my addr 172.21.15.87:0/4068523835 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.864+0000 7fb62ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 0x7fb6301030b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34768/0 (socket says 172.21.15.87:34768) 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62e59c640 1 -- 172.21.15.87:0/4068523835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630101790 msgr2=0x7fb630102b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62e59c640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630101790 0x7fb630102b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62e59c640 1 -- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 msgr2=0x7fb6301030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62e59c640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 0x7fb6301030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62e59c640 1 -- 172.21.15.87:0/4068523835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb630103970 con 0x7fb6301005a0 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62ed9d640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 0x7fb6301030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62dd9b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630101790 0x7fb630102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb62e59c640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 0x7fb6301044e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb6200324c0 tx=0x7fb620032f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620041020 con 0x7fb6301005a0 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6301a1b40 con 0x7fb6301005a0 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb620034cd0 con 0x7fb6301005a0 2024-03-21T08:56:32.863 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6301a2050 con 0x7fb6301005a0 2024-03-21T08:56:32.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.865+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb620039040 con 0x7fb6301005a0 2024-03-21T08:56:32.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.866+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6301a2220 con 0x7fb6301005a0 2024-03-21T08:56:32.864 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.867+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb620044020 con 0x7fb6301005a0 2024-03-21T08:56:32.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.868+0000 7fb61f7fe640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb6180a2f20 0x7fb6180a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:32.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.868+0000 7fb62dd9b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb6180a2f20 0x7fb6180a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:32.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.868+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb6200f1620 con 0x7fb6301005a0 2024-03-21T08:56:32.865 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.868+0000 7fb62dd9b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb6180a2f20 0x7fb6180a53e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb6240095f0 tx=0x7fb624005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:32.871 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:32.876+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb62004b020 con 0x7fb6301005a0 2024-03-21T08:56:33.047 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.052+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6301a2510 con 0x7fb6301005a0 2024-03-21T08:56:33.048 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.053+0000 7fb61f7fe640 1 -- 172.21.15.87:0/4068523835 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6200be4b0 con 0x7fb6301005a0 2024-03-21T08:56:33.048 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:56:33.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb6180a2f20 msgr2=0x7fb6180a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb6180a2f20 0x7fb6180a53e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fb6240095f0 tx=0x7fb624005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:33.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 msgr2=0x7fb6301044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 0x7fb6301044e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb6200324c0 tx=0x7fb620032f40 comp rx=0 tx=0).stop 2024-03-21T08:56:33.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 shutdown_connections 2024-03-21T08:56:33.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb6180a2f20 0x7fb6180a53e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb630102150 0x7fb6301030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb630101790 0x7fb630102b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 --2- 172.21.15.87:0/4068523835 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb6301005a0 0x7fb6301044e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 >> 172.21.15.87:0/4068523835 conn(0x7fb6300fbb30 msgr2=0x7fb6300fd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 shutdown_connections 2024-03-21T08:56:33.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.055+0000 7fb634d5b640 1 -- 172.21.15.87:0/4068523835 wait complete. 2024-03-21T08:56:33.060 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:32 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/774663823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:33.060 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:32 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1510364376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:33.060 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:32 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4075241862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:33.061 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:56:33.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 -- 172.21.15.87:0/2251556558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 msgr2=0x7fc3e0106ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 --2- 172.21.15.87:0/2251556558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0106ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc3d0009860 tx=0x7fc3d002bb70 comp rx=0 tx=0).stop 2024-03-21T08:56:33.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 -- 172.21.15.87:0/2251556558 shutdown_connections 2024-03-21T08:56:33.168 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 --2- 172.21.15.87:0/2251556558 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3e01073e0 0x7fc3e01097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 --2- 172.21.15.87:0/2251556558 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0106ea0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 --2- 172.21.15.87:0/2251556558 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3e0102180 0x7fc3e0104570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 -- 172.21.15.87:0/2251556558 >> 172.21.15.87:0/2251556558 conn(0x7fc3e00fbb30 msgr2=0x7fc3e00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 -- 172.21.15.87:0/2251556558 shutdown_connections 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.170+0000 7fc3e7568640 1 -- 172.21.15.87:0/2251556558 wait complete. 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 Processor -- start 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 -- start start 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3e0102180 0x7fc3e0197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3e01073e0 0x7fc3e019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e010d300 con 0x7fc3e0104ab0 2024-03-21T08:56:33.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e010d180 con 0x7fc3e0102180 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e7568640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e010d480 con 0x7fc3e01073e0 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34784/0 (socket says 172.21.15.87:34784) 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 -- 172.21.15.87:0/244491415 learned_addr learned my addr 172.21.15.87:0/244491415 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e5304640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3e0102180 0x7fc3e0197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e5b05640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3e01073e0 0x7fc3e019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 -- 172.21.15.87:0/244491415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3e01073e0 msgr2=0x7fc3e019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3e01073e0 0x7fc3e019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 -- 172.21.15.87:0/244491415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3e0102180 msgr2=0x7fc3e0197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3e0102180 0x7fc3e0197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.171+0000 7fc3e4b03640 1 -- 172.21.15.87:0/244491415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e019f650 con 0x7fc3e0104ab0 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.172+0000 7fc3e4b03640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0197f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3d002c0b0 tx=0x7fc3d0032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:33.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.172+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0041020 con 0x7fc3e0104ab0 2024-03-21T08:56:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.172+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc3d002c9f0 con 0x7fc3e0104ab0 2024-03-21T08:56:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.172+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e019f8e0 con 0x7fc3e0104ab0 2024-03-21T08:56:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.172+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d0039040 con 0x7fc3e0104ab0 2024-03-21T08:56:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.172+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e019fdd0 con 0x7fc3e0104ab0 2024-03-21T08:56:33.171 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.173+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e00fd620 con 0x7fc3e0104ab0 2024-03-21T08:56:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.174+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc3d0044020 con 0x7fc3e0104ab0 2024-03-21T08:56:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.175+0000 7fc3d67fc640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc3cc0a2f20 0x7fc3cc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.175+0000 7fc3e5304640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc3cc0a2f20 0x7fc3cc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.175+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc3d00f1090 con 0x7fc3e0104ab0 2024-03-21T08:56:33.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.175+0000 7fc3e5304640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc3cc0a2f20 0x7fc3cc0a53e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009bd0 tx=0x7fc3dc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:33.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.179+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc3d004b020 con 0x7fc3e0104ab0 2024-03-21T08:56:33.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:32 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/774663823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:33.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:32 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1510364376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:33.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:32 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4075241862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:33.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:32 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/774663823' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:33.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:32 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1510364376' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:33.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:32 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4075241862' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:33.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.351+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc3e002d200 con 0x7fc3e0104ab0 2024-03-21T08:56:33.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.351+0000 7fc3d67fc640 1 -- 172.21.15.87:0/244491415 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc3d00bdef0 con 0x7fc3e0104ab0 2024-03-21T08:56:33.347 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-03-21T08:56:33.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc3cc0a2f20 msgr2=0x7fc3cc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc3cc0a2f20 0x7fc3cc0a53e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009bd0 tx=0x7fc3dc005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 msgr2=0x7fc3e0197f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0197f70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3d002c0b0 tx=0x7fc3d0032430 comp rx=0 tx=0).stop 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 shutdown_connections 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc3cc0a2f20 0x7fc3cc0a53e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc3e01073e0 0x7fc3e019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc3e0104ab0 0x7fc3e0197f70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 --2- 172.21.15.87:0/244491415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc3e0102180 0x7fc3e0197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 >> 172.21.15.87:0/244491415 conn(0x7fc3e00fbb30 msgr2=0x7fc3e0102a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 shutdown_connections 2024-03-21T08:56:33.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.353+0000 7fc3e7568640 1 -- 172.21.15.87:0/244491415 wait complete. 2024-03-21T08:56:33.360 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:56:33.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 -- 172.21.15.87:0/3381579631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1028a0 msgr2=0x7f316c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 --2- 172.21.15.87:0/3381579631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1028a0 0x7f316c102d20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f315c009860 tx=0x7f315c02bb70 comp rx=0 tx=0).stop 2024-03-21T08:56:33.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 -- 172.21.15.87:0/3381579631 shutdown_connections 2024-03-21T08:56:33.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 --2- 172.21.15.87:0/3381579631 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f316c103260 0x7f316c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 --2- 172.21.15.87:0/3381579631 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1028a0 0x7f316c102d20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.466 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 --2- 172.21.15.87:0/3381579631 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c1016b0 0x7f316c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 -- 172.21.15.87:0/3381579631 >> 172.21.15.87:0/3381579631 conn(0x7f316c0fcc40 msgr2=0x7f316c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 -- 172.21.15.87:0/3381579631 shutdown_connections 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.468+0000 7f3173f37640 1 -- 172.21.15.87:0/3381579631 wait complete. 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 Processor -- start 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 -- start start 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1016b0 0x7f316c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f316c1028a0 0x7f316c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 0x7f316c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10d310 con 0x7f316c103260 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10d190 con 0x7f316c1028a0 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3173f37640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c10d490 con 0x7f316c1016b0 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 0x7f316c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.467 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 0x7f316c19ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34792/0 (socket says 172.21.15.87:34792) 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 -- 172.21.15.87:0/592354890 learned_addr learned my addr 172.21.15.87:0/592354890 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f3171cd3640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1016b0 0x7f316c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31714d2640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f316c1028a0 0x7f316c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 -- 172.21.15.87:0/592354890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1016b0 msgr2=0x7f316c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1016b0 0x7f316c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 -- 172.21.15.87:0/592354890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f316c1028a0 msgr2=0x7f316c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f316c1028a0 0x7f316c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.469+0000 7f31724d4640 1 -- 172.21.15.87:0/592354890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c19f600 con 0x7f316c103260 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.470+0000 7f31724d4640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 0x7f316c19ef90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f316400eb10 tx=0x7f316400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.470+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164004b20 con 0x7f316c103260 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.470+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3164004cc0 con 0x7f316c103260 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.470+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c19f8f0 con 0x7f316c103260 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.470+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3164010760 con 0x7f316c103260 2024-03-21T08:56:33.468 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.470+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c19fdc0 con 0x7f316c103260 2024-03-21T08:56:33.469 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.471+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c108a50 con 0x7f316c103260 2024-03-21T08:56:33.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.472+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3164005020 con 0x7f316c103260 2024-03-21T08:56:33.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.472+0000 7f3162ffd640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31580a2f20 0x7f31580a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.472+0000 7f3171cd3640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31580a2f20 0x7f31580a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.472+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f31640cf910 con 0x7f316c103260 2024-03-21T08:56:33.470 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.473+0000 7f3171cd3640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31580a2f20 0x7f31580a53e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3168009530 tx=0x7f3168009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:33.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.477+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f316409c770 con 0x7f316c103260 2024-03-21T08:56:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.655+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f316c0622a0 con 0x7f316c103260 2024-03-21T08:56:33.651 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.655+0000 7f3162ffd640 1 -- 172.21.15.87:0/592354890 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f316409c070 con 0x7f316c103260 2024-03-21T08:56:33.651 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31580a2f20 msgr2=0x7f31580a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31580a2f20 0x7f31580a53e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3168009530 tx=0x7f3168009260 comp rx=0 tx=0).stop 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 msgr2=0x7f316c19ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 0x7f316c19ef90 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f316400eb10 tx=0x7f316400c2d0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 shutdown_connections 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f31580a2f20 0x7f31580a53e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f316c103260 0x7f316c19ef90 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f316c1028a0 0x7f316c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.654 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 --2- 172.21.15.87:0/592354890 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f316c1016b0 0x7f316c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 >> 172.21.15.87:0/592354890 conn(0x7f316c0fcc40 msgr2=0x7f316c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 shutdown_connections 2024-03-21T08:56:33.655 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.657+0000 7f3173f37640 1 -- 172.21.15.87:0/592354890 wait complete. 2024-03-21T08:56:33.664 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 -- 172.21.15.87:0/3481826198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f201021c0 msgr2=0x7f7f201045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/3481826198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f201021c0 0x7f7f201045b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f7f14009900 tx=0x7f7f1402bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 -- 172.21.15.87:0/3481826198 shutdown_connections 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/3481826198 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f20107420 0x7f7f20109870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/3481826198 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f20104af0 0x7f7f20106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/3481826198 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f201021c0 0x7f7f201045b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 -- 172.21.15.87:0/3481826198 >> 172.21.15.87:0/3481826198 conn(0x7f7f200fbbd0 msgr2=0x7f7f200fdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.768 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 -- 172.21.15.87:0/3481826198 shutdown_connections 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 -- 172.21.15.87:0/3481826198 wait complete. 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 Processor -- start 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 -- start start 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.770+0000 7f7f26c0b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 0x7f7f20197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 0x7f7f20197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f20107420 0x7f7f2019efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2010d360 con 0x7f7f20104af0 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2010d1e0 con 0x7f7f201021c0 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2010d4e0 con 0x7f7f20107420 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 0x7f7f20197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f249a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 0x7f7f20197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f249a7640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 0x7f7f20197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:40784/0 (socket says 172.21.15.87:40784) 2024-03-21T08:56:33.769 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f249a7640 1 -- 172.21.15.87:0/310260423 learned_addr learned my addr 172.21.15.87:0/310260423 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 0x7f7f20197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34806/0 (socket says 172.21.15.87:34806) 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f251a8640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f20107420 0x7f7f2019efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 -- 172.21.15.87:0/310260423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f20107420 msgr2=0x7f7f2019efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f20107420 0x7f7f2019efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 -- 172.21.15.87:0/310260423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 msgr2=0x7f7f20197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 0x7f7f20197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 -- 172.21.15.87:0/310260423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2019f620 con 0x7f7f20104af0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f249a7640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 0x7f7f20197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1ffff640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 0x7f7f20197f40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9c0 tx=0x7f7f0c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c00cd80 con 0x7f7f20104af0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f2019f910 con 0x7f7f20104af0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f0c0035f0 con 0x7f7f20104af0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c010670 con 0x7f7f20104af0 2024-03-21T08:56:33.770 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.771+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f2019fe60 con 0x7f7f20104af0 2024-03-21T08:56:33.771 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.772+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f200fd6c0 con 0x7f7f20104af0 2024-03-21T08:56:33.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.773+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f0c0037a0 con 0x7f7f20104af0 2024-03-21T08:56:33.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.774+0000 7f7f1dffb640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f100a2f20 0x7f7f100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:33.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.774+0000 7f7f249a7640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f100a2f20 0x7f7f100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:33.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.774+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7f0c0cf680 con 0x7f7f20104af0 2024-03-21T08:56:33.772 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.774+0000 7f7f249a7640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f100a2f20 0x7f7f100a53e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000c00 tx=0x7f7f1402c880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:33.774 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.778+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f0c09c4e0 con 0x7f7f20104af0 2024-03-21T08:56:33.987 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.991+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7f2002d070 con 0x7f7f20104af0 2024-03-21T08:56:33.988 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.992+0000 7f7f1dffb640 1 -- 172.21.15.87:0/310260423 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11460 (secure 0 0 0) 0x7f7f0c09bde0 con 0x7f7f20104af0 2024-03-21T08:56:33.988 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:33.988 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:55:12.955598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24451,24482,24425,24487,24403]},"id":1}]} 2024-03-21T08:56:33.988 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:56:33.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f100a2f20 msgr2=0x7f7f100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f100a2f20 0x7f7f100a53e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f7f14000c00 tx=0x7f7f1402c880 comp rx=0 tx=0).stop 2024-03-21T08:56:33.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 msgr2=0x7f7f20197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 0x7f7f20197f40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c00e9c0 tx=0x7f7f0c00ee80 comp rx=0 tx=0).stop 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 shutdown_connections 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f100a2f20 0x7f7f100a53e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f20107420 0x7f7f2019efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f20104af0 0x7f7f20197f40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 --2- 172.21.15.87:0/310260423 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f201021c0 0x7f7f20197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 >> 172.21.15.87:0/310260423 conn(0x7f7f200fbbd0 msgr2=0x7f7f20102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 shutdown_connections 2024-03-21T08:56:33.991 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:33.994+0000 7f7f26c0b640 1 -- 172.21.15.87:0/310260423 wait complete. 2024-03-21T08:56:34.001 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 -- 172.21.15.87:0/1303264925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c104a60 msgr2=0x7f128c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 --2- 172.21.15.87:0/1303264925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c104a60 0x7f128c106e50 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f127c009900 tx=0x7f127c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 -- 172.21.15.87:0/1303264925 shutdown_connections 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 --2- 172.21.15.87:0/1303264925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 0x7f128c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 --2- 172.21.15.87:0/1303264925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c104a60 0x7f128c106e50 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 --2- 172.21.15.87:0/1303264925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c102130 0x7f128c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 -- 172.21.15.87:0/1303264925 >> 172.21.15.87:0/1303264925 conn(0x7f128c0fbb40 msgr2=0x7f128c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 -- 172.21.15.87:0/1303264925 shutdown_connections 2024-03-21T08:56:34.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.107+0000 7f1294559640 1 -- 172.21.15.87:0/1303264925 wait complete. 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 Processor -- start 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 -- start start 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 0x7f128c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c104a60 0x7f128c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 0x7f128c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c10d330 con 0x7f128c102130 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c10d1b0 con 0x7f128c104a60 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1294559640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c10d4b0 con 0x7f128c107390 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 0x7f128c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 0x7f128c1979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34816/0 (socket says 172.21.15.87:34816) 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 -- 172.21.15.87:0/2729180199 learned_addr learned my addr 172.21.15.87:0/2729180199 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1291af4640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c104a60 0x7f128c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1292af6640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 0x7f128c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 -- 172.21.15.87:0/2729180199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 msgr2=0x7f128c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 0x7f128c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 -- 172.21.15.87:0/2729180199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c104a60 msgr2=0x7f128c197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c104a60 0x7f128c197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f12922f5640 1 -- 172.21.15.87:0/2729180199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c19f600 con 0x7f128c102130 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.108+0000 7f1292af6640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 0x7f128c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.109+0000 7f12922f5640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 0x7f128c1979e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f128800e9f0 tx=0x7f128800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.109+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f128800cd80 con 0x7f128c102130 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.109+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1288010910 con 0x7f128c102130 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.109+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c19f8f0 con 0x7f128c102130 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.109+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288018990 con 0x7f128c102130 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.109+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c19fe10 con 0x7f128c102130 2024-03-21T08:56:34.108 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.110+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c0fd630 con 0x7f128c102130 2024-03-21T08:56:34.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.111+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1288010ac0 con 0x7f128c102130 2024-03-21T08:56:34.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.111+0000 7f12837fe640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12780a2f20 0x7f12780a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.111+0000 7f1291af4640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12780a2f20 0x7f12780a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.111+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f12880cf760 con 0x7f128c102130 2024-03-21T08:56:34.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.112+0000 7f1291af4640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12780a2f20 0x7f12780a53e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f127c000c00 tx=0x7f127c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.111 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.116+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f128809c5c0 con 0x7f128c102130 2024-03-21T08:56:34.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:33 smithi105 ceph-mon[35993]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:56:34.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:33 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4068523835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:33 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/244491415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:33 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/592354890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:33 smithi100 ceph-mon[36306]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:56:34.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:33 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4068523835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:33 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/244491415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:33 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/592354890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.291 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.295+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f128c02d070 con 0x7f128c102130 2024-03-21T08:56:34.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.297+0000 7f12837fe640 1 -- 172.21.15.87:0/2729180199 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7f128809bec0 con 0x7f128c102130 2024-03-21T08:56:34.293 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:34.293 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12780a2f20 msgr2=0x7f12780a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12780a2f20 0x7f12780a53e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f127c000c00 tx=0x7f127c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 msgr2=0x7f128c1979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 0x7f128c1979e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f128800e9f0 tx=0x7f128800eeb0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 shutdown_connections 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12780a2f20 0x7f12780a53e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f128c107390 0x7f128c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f128c104a60 0x7f128c197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.295 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 --2- 172.21.15.87:0/2729180199 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f128c102130 0x7f128c1979e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 >> 172.21.15.87:0/2729180199 conn(0x7f128c0fbb40 msgr2=0x7f128c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 shutdown_connections 2024-03-21T08:56:34.296 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.299+0000 7f1294559640 1 -- 172.21.15.87:0/2729180199 wait complete. 2024-03-21T08:56:34.305 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:33 smithi087 ceph-mon[32385]: pgmap v335: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 10 op/s 2024-03-21T08:56:34.305 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:33 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4068523835' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:33 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/244491415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.306 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:33 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/592354890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:34.307 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:56:34.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.412+0000 7f532d268640 1 -- 172.21.15.87:0/3001455625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53280ff9b0 msgr2=0x7f53280ffdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.412+0000 7f532d268640 1 --2- 172.21.15.87:0/3001455625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53280ff9b0 0x7f53280ffdd0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5318009930 tx=0x7f531802bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:34.411 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- 172.21.15.87:0/3001455625 shutdown_connections 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 --2- 172.21.15.87:0/3001455625 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328103340 0x7f5328109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 --2- 172.21.15.87:0/3001455625 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5328102980 0x7f5328102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 --2- 172.21.15.87:0/3001455625 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f53280ff9b0 0x7f53280ffdd0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- 172.21.15.87:0/3001455625 >> 172.21.15.87:0/3001455625 conn(0x7f53280fb2e0 msgr2=0x7f53280fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- 172.21.15.87:0/3001455625 shutdown_connections 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- 172.21.15.87:0/3001455625 wait complete. 2024-03-21T08:56:34.412 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 Processor -- start 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- start start 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53280ff9b0 0x7f5328197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328102980 0x7f5328197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 0x7f532819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532810d360 con 0x7f5328103340 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532810d1e0 con 0x7f53280ff9b0 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.413+0000 7f532d268640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f532810d4e0 con 0x7f5328102980 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 0x7f532819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 0x7f532819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34820/0 (socket says 172.21.15.87:34820) 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 -- 172.21.15.87:0/2776658931 learned_addr learned my addr 172.21.15.87:0/2776658931 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53267fc640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328102980 0x7f5328197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f5326ffd640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53280ff9b0 0x7f5328197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 -- 172.21.15.87:0/2776658931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328102980 msgr2=0x7f5328197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328102980 0x7f5328197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 -- 172.21.15.87:0/2776658931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53280ff9b0 msgr2=0x7f5328197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53280ff9b0 0x7f5328197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 -- 172.21.15.87:0/2776658931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f532819f650 con 0x7f5328103340 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f53277fe640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 0x7f532819efe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f532000eb10 tx=0x7f532000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5320004b20 con 0x7f5328103340 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f532819f940 con 0x7f5328103340 2024-03-21T08:56:34.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5320004cc0 con 0x7f5328103340 2024-03-21T08:56:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5320010760 con 0x7f5328103340 2024-03-21T08:56:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.414+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f532819fe40 con 0x7f5328103340 2024-03-21T08:56:34.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.415+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5328108b30 con 0x7f5328103340 2024-03-21T08:56:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.416+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5320005020 con 0x7f5328103340 2024-03-21T08:56:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.417+0000 7f530ffff640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f53100a2f20 0x7f53100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.417+0000 7f5326ffd640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f53100a2f20 0x7f53100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.417+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f53200cf910 con 0x7f5328103340 2024-03-21T08:56:34.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.417+0000 7f5326ffd640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f53100a2f20 0x7f53100a53e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f531802c7b0 tx=0x7f5318000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.421+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f532009c770 con 0x7f5328103340 2024-03-21T08:56:34.631 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.636+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f532802d070 con 0x7f5328103340 2024-03-21T08:56:34.632 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.636+0000 7f530ffff640 1 -- 172.21.15.87:0/2776658931 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11460 (secure 0 0 0) 0x7f532009c070 con 0x7f5328103340 2024-03-21T08:56:34.633 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:34.633 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:55:12.955598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24451,24482,24425,24487,24403]},"id":1}]} 2024-03-21T08:56:34.633 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.638+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f53100a2f20 msgr2=0x7f53100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.638+0000 7f532d268640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f53100a2f20 0x7f53100a53e0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f531802c7b0 tx=0x7f5318000bc0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.638+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 msgr2=0x7f532819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.638+0000 7f532d268640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 0x7f532819efe0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f532000eb10 tx=0x7f532000c2d0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 shutdown_connections 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f53100a2f20 0x7f53100a53e0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5328103340 0x7f532819efe0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.635 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5328102980 0x7f5328197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 --2- 172.21.15.87:0/2776658931 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f53280ff9b0 0x7f5328197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 >> 172.21.15.87:0/2776658931 conn(0x7f53280fb2e0 msgr2=0x7f5328105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 shutdown_connections 2024-03-21T08:56:34.636 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.639+0000 7f532d268640 1 -- 172.21.15.87:0/2776658931 wait complete. 2024-03-21T08:56:34.646 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T08:55:06.567291+0000', 'modified': '2024-03-21T08:55:12.955598+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24482, 'mds_2': 24451, 'mds_3': 24425, 'mds_4': 24403}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6839/3694307942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 3694307942}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 3694307942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14646': {'gid': 14646, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6841/2007269086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6840', 'nonce': 2007269086}, {'type': 'v1', 'addr': '172.21.15.87:6841', 'nonce': 2007269086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6833/1637520984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 1637520984}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 1637520984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.105:6833/500134057', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 500134057}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 500134057}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.100:6835/2548903221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2548903221}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2548903221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.105:6835/1031026542', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1031026542}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1031026542}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6837/2948361419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 2948361419}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 2948361419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/2656551527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 2656551527}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2656551527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/214101858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 214101858}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 214101858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6839/892092628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 892092628}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 892092628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24487, 'qdb_cluster': [24451, 24482, 24425, 24487, 24403]} 2024-03-21T08:56:34.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-21T08:56:34.647 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-21T08:56:34.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 -- 172.21.15.87:0/2480709124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1028a0 msgr2=0x7f911c102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 --2- 172.21.15.87:0/2480709124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1028a0 0x7f911c102d20 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f9104009900 tx=0x7f910402bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:34.751 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 -- 172.21.15.87:0/2480709124 shutdown_connections 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 --2- 172.21.15.87:0/2480709124 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c103260 0x7f911c109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 --2- 172.21.15.87:0/2480709124 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c1028a0 0x7f911c102d20 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 --2- 172.21.15.87:0/2480709124 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c1016b0 0x7f911c101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 -- 172.21.15.87:0/2480709124 >> 172.21.15.87:0/2480709124 conn(0x7f911c0fcc40 msgr2=0x7f911c0ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 -- 172.21.15.87:0/2480709124 shutdown_connections 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.753+0000 7f91222dc640 1 -- 172.21.15.87:0/2480709124 wait complete. 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 Processor -- start 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 -- start start 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c1016b0 0x7f911c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 0x7f911c197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c103260 0x7f911c19ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c10d310 con 0x7f911c103260 2024-03-21T08:56:34.752 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c10d190 con 0x7f911c1028a0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f91222dc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c10d490 con 0x7f911c1016b0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 0x7f911c197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c1016b0 0x7f911c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 0x7f911c197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:40846/0 (socket says 172.21.15.87:40846) 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911b7fe640 1 -- 172.21.15.87:0/1771961299 learned_addr learned my addr 172.21.15.87:0/1771961299 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f9120879640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c103260 0x7f911c19ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911b7fe640 1 -- 172.21.15.87:0/1771961299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c1016b0 msgr2=0x7f911c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911b7fe640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c1016b0 0x7f911c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.754+0000 7f911b7fe640 1 -- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c103260 msgr2=0x7f911c19ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f911b7fe640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c103260 0x7f911c19ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f911b7fe640 1 -- 172.21.15.87:0/1771961299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c19f600 con 0x7f911c1028a0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f911b7fe640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 0x7f911c197f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9104009230 tx=0x7f910402c650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f91197fa640 1 -- 172.21.15.87:0/1771961299 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9104041070 con 0x7f911c1028a0 2024-03-21T08:56:34.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c19f890 con 0x7f911c1028a0 2024-03-21T08:56:34.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f91197fa640 1 -- 172.21.15.87:0/1771961299 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9104004450 con 0x7f911c1028a0 2024-03-21T08:56:34.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f91197fa640 1 -- 172.21.15.87:0/1771961299 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910404b060 con 0x7f911c1028a0 2024-03-21T08:56:34.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.755+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c19fd80 con 0x7f911c1028a0 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f91197fa640 1 -- 172.21.15.87:0/1771961299 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f9104044070 con 0x7f911c1028a0 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 Processor -- start 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 -- start start 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f5740 0x562ffb5f5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 0x562ffb5fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5fd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb55ac20 con 0x562ffb5f6130 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f91197fa640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91080a2f20 0x7f91080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb503a40 con 0x562ffb5f5740 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.757+0000 7f9124fa4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb556cc0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 0x562ffb5fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f9120879640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f5740 0x562ffb5f5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 0x562ffb5fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34854/0 (socket says 172.21.15.87:34854) 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 -- 172.21.15.87:0/3255119141 learned_addr learned my addr 172.21.15.87:0/3255119141 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911b7fe640 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5fd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f9120879640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91080a2f20 0x7f91080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f91197fa640 1 -- 172.21.15.87:0/1771961299 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f91040f12a0 con 0x7f911c1028a0 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 -- 172.21.15.87:0/3255119141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 msgr2=0x562ffb5fd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5fd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 -- 172.21.15.87:0/3255119141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f5740 msgr2=0x562ffb5f5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f5740 0x562ffb5f5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 -- 172.21.15.87:0/3255119141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ffb525f90 con 0x562ffb5f6130 2024-03-21T08:56:34.761 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f911bfff640 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 0x562ffb5fac30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ca0 tx=0x7f910c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a4ceec1986ab71f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f9120879640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91080a2f20 0x7f91080a53e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x562ffb5f6990 tx=0x7f9114009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f91127fc640 1 -- 172.21.15.87:0/3255119141 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c004220 con 0x562ffb5f6130 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f91127fc640 1 -- 172.21.15.87:0/3255119141 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910c0043c0 con 0x562ffb5f6130 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.758+0000 7f91127fc640 1 -- 172.21.15.87:0/3255119141 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c019930 con 0x562ffb5f6130 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 -- 172.21.15.87:0/3255119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 msgr2=0x562ffb5fac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 0x562ffb5fac30 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f910c009ca0 tx=0x7f910c011810 comp rx=0 tx=0).stop 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 -- 172.21.15.87:0/3255119141 shutdown_connections 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5fd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f6130 0x562ffb5fac30 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 --2- 172.21.15.87:0/3255119141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f5740 0x562ffb5f5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 -- 172.21.15.87:0/3255119141 >> 172.21.15.87:0/3255119141 conn(0x562ffb5bda60 msgr2=0x562ffb5f10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 -- 172.21.15.87:0/3255119141 shutdown_connections 2024-03-21T08:56:34.762 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 -- 172.21.15.87:0/3255119141 wait complete. 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 -- start start 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.759+0000 7f9124fa4740 1 Processor -- start 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f5740 0x562ffb604050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb604590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 0x562ffb60b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb5fe250 con 0x562ffb5fb2a0 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb516e90 con 0x562ffb5f6130 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb5fe3d0 con 0x562ffb5f5740 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 10 client.0 _ll_register_callbacks cb 0x562ffb26eac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 0x562ffb60b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f911bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f5740 0x562ffb604050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f911b7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb604590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.763 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 0x562ffb60b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34870/0 (socket says 172.21.15.87:34870) 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f911bfff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f5740 0x562ffb604050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58678/0 (socket says 172.21.15.87:58678) 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 -- 172.21.15.87:0/2638913041 learned_addr learned my addr 172.21.15.87:0/2638913041 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 -- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f5740 msgr2=0x562ffb604050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f5740 0x562ffb604050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 -- 172.21.15.87:0/2638913041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 msgr2=0x562ffb604590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb604590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 -- 172.21.15.87:0/2638913041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ffb5239d0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f911b7fe640 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb604590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9120879640 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 0x562ffb60b5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9114011ee0 tx=0x7f9114004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52809a0d280289fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f90fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2638913041 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911401e410 con 0x562ffb5fb2a0 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f910c0043c0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.764 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2638913041 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f911401e5b0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2638913041 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f911401e7b0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 10 client.15072 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.760+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ffb26f2d0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2638913041 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910c0043c0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2638913041 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f911401d070 con 0x562ffb5fb2a0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f9124fa4740 10 client.15072 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f9124fa4740 10 client.15072 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f9124fa4740 20 client.15072 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb6108d0 0x562ffb60f910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f9124fa4740 4 client.15072 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 --> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562ffb568160 con 0x562ffb6108d0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f911bfff640 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb6108d0 0x562ffb60f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f911bfff640 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb6108d0 0x562ffb60f910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f73e410f534d8cba server_cookie=9cd6969c251c0b29 in_seq=0 out_seq=0 2024-03-21T08:56:34.765 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.761+0000 7f90fb7fe640 10 client.15072 ms_handle_connect on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.762+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2638913041 <== mds.4 v2:172.21.15.105:6832/500134057 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x562ffb568160 con 0x562ffb6108d0 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.762+0000 7f90fb7fe640 10 client.15072 handle_command_reply: tid=0 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.762+0000 7f9124fa4740 1 client.15072 shutdown 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 msgr2=0x562ffb60b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 0x562ffb60b5d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9114011ee0 tx=0x7f9114004ab0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 shutdown_connections 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb6108d0 0x562ffb60f910 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5fb2a0 0x562ffb60b5d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb604590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f90fb7fe640 0 client.15072 ms_handle_reset on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2638913041 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f5740 0x562ffb604050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 >> 172.21.15.87:0/2638913041 conn(0x562ffb5bda60 msgr2=0x562ffb5f02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.766 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 shutdown_connections 2024-03-21T08:56:34.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 1 -- 172.21.15.87:0/2638913041 wait complete. 2024-03-21T08:56:34.767 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.763+0000 7f9124fa4740 20 client.15072 trim_cache size 0 max 16384 2024-03-21T08:56:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 Processor -- start 2024-03-21T08:56:34.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 -- start start 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb603a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb60b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb6100f0 con 0x562ffb5f5740 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb610750 con 0x562ffb5f6130 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9124fa4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb610db0 con 0x562ffb5fb2a0 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f911bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb603a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f911bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb603a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34884/0 (socket says 172.21.15.87:34884) 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f911bfff640 1 -- 172.21.15.87:0/2171117605 learned_addr learned my addr 172.21.15.87:0/2171117605 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f911b7fe640 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb60b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.779+0000 7f9120879640 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f911bfff640 1 -- 172.21.15.87:0/2171117605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 msgr2=0x562ffb5f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f911bfff640 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f911bfff640 1 -- 172.21.15.87:0/2171117605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 msgr2=0x562ffb60b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f911bfff640 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb60b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f911bfff640 1 -- 172.21.15.87:0/2171117605 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562ffb3396b0 con 0x562ffb5f5740 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f911bfff640 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb603a30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f910c014040 tx=0x7f910c011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24a97e1354ace1c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9111ffb640 1 -- 172.21.15.87:0/2171117605 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c02f020 con 0x562ffb5f5740 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9111ffb640 1 -- 172.21.15.87:0/2171117605 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910c02b2c0 con 0x562ffb5f5740 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9111ffb640 1 -- 172.21.15.87:0/2171117605 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c003b00 con 0x562ffb5f5740 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 -- 172.21.15.87:0/2171117605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 msgr2=0x562ffb603a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb603a30 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f910c014040 tx=0x7f910c011810 comp rx=0 tx=0).stop 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 -- 172.21.15.87:0/2171117605 shutdown_connections 2024-03-21T08:56:34.782 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5fb2a0 0x562ffb5f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5f6130 0x562ffb60b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2171117605 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb603a30 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.780+0000 7f9124fa4740 1 -- 172.21.15.87:0/2171117605 >> 172.21.15.87:0/2171117605 conn(0x562ffb5bda60 msgr2=0x562ffb603320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 -- 172.21.15.87:0/2171117605 shutdown_connections 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 -- 172.21.15.87:0/2171117605 wait complete. 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 -- start start 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 Processor -- start 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb60b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f6130 0x562ffb607df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5fb2a0 0x562ffb608330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb611a70 con 0x562ffb5f5740 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb611410 con 0x562ffb5fb2a0 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x562ffb6120d0 con 0x562ffb5f6130 2024-03-21T08:56:34.783 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9124fa4740 10 client.0 _ll_register_callbacks cb 0x7f9114011da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f911bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb60b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f911bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb60b300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34894/0 (socket says 172.21.15.87:34894) 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f911bfff640 1 -- 172.21.15.87:0/2539437902 learned_addr learned my addr 172.21.15.87:0/2539437902 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f9120879640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5fb2a0 0x562ffb608330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.781+0000 7f911b7fe640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f6130 0x562ffb607df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f911bfff640 1 -- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f6130 msgr2=0x562ffb607df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f911bfff640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f6130 0x562ffb607df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f911bfff640 1 -- 172.21.15.87:0/2539437902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5fb2a0 msgr2=0x562ffb608330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f911bfff640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5fb2a0 0x562ffb608330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f911bfff640 1 -- 172.21.15.87:0/2539437902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562ffb6089a0 con 0x562ffb5f5740 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f911bfff640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb60b300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f910c0058c0 tx=0x7f910c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92078a5a91fcaefe server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f90fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:34.784 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2539437902 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c003c10 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2539437902 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f910c005140 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f910c02b2c0 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2539437902 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f910c046050 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 10 client.15078 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x562ffb66ca10 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2539437902 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f910c02b2c0 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2539437902 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f910c02bc00 con 0x562ffb5f5740 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 10 client.15078 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 10 client.15078 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 20 client.15078 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb670a10 0x562ffb66cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 4 client.15078 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.782+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 --> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562ffb5feb70 con 0x562ffb670a10 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f911b7fe640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb670a10 0x562ffb66cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.785 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f911b7fe640 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb670a10 0x562ffb66cbe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=cc259b5bc1621cd7 server_cookie=1fb2d2e1ab9dcfb in_seq=0 out_seq=0 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f90fb7fe640 10 client.15078 ms_handle_connect on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f90fb7fe640 1 -- 172.21.15.87:0/2539437902 <== mds.4 v2:172.21.15.105:6832/500134057 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x562ffb5feb70 con 0x562ffb670a10 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f90fb7fe640 10 client.15078 handle_command_reply: tid=0 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 client.15078 shutdown 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 msgr2=0x562ffb60b300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb60b300 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f910c0058c0 tx=0x7f910c009c30 comp rx=0 tx=0).stop 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 shutdown_connections 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x562ffb670a10 0x562ffb66cbe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x562ffb5fb2a0 0x562ffb608330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f90fb7fe640 0 client.15078 ms_handle_reset on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x562ffb5f6130 0x562ffb607df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.786 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 --2- 172.21.15.87:0/2539437902 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x562ffb5f5740 0x562ffb60b300 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 >> 172.21.15.87:0/2539437902 conn(0x562ffb5bda60 msgr2=0x562ffb5f02f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 shutdown_connections 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 1 -- 172.21.15.87:0/2539437902 wait complete. 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.783+0000 7f9124fa4740 20 client.15078 trim_cache size 0 max 16384 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 4, 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24403, 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:34.787 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:34.788 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 82.833176406000007, 2024-03-21T08:56:34.788 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:34.788 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:34.788 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:34.788 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 125.72641194800001 2024-03-21T08:56:34.788 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91080a2f20 msgr2=0x7f91080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91080a2f20 0x7f91080a53e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x562ffb5f6990 tx=0x7f9114009500 comp rx=0 tx=0).stop 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 msgr2=0x7f911c197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 0x7f911c197f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9104009230 tx=0x7f910402c650 comp rx=0 tx=0).stop 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 shutdown_connections 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f91080a2f20 0x7f91080a53e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f911c103260 0x7f911c19ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f911c1028a0 0x7f911c197f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 --2- 172.21.15.87:0/1771961299 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f911c1016b0 0x7f911c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 >> 172.21.15.87:0/1771961299 conn(0x7f911c0fcc40 msgr2=0x7f911c105c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 shutdown_connections 2024-03-21T08:56:34.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.784+0000 7f91222dc640 1 -- 172.21.15.87:0/1771961299 wait complete. 2024-03-21T08:56:34.791 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-21T08:56:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 -- 172.21.15.87:0/232670861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098104a60 msgr2=0x7f3098106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 --2- 172.21.15.87:0/232670861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098104a60 0x7f3098106e50 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3088009900 tx=0x7f308802bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 -- 172.21.15.87:0/232670861 shutdown_connections 2024-03-21T08:56:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 --2- 172.21.15.87:0/232670861 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098107390 0x7f30981097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.895 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 --2- 172.21.15.87:0/232670861 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098104a60 0x7f3098106e50 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 --2- 172.21.15.87:0/232670861 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3098102130 0x7f3098104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 -- 172.21.15.87:0/232670861 >> 172.21.15.87:0/232670861 conn(0x7f30980fbb40 msgr2=0x7f30980fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 -- 172.21.15.87:0/232670861 shutdown_connections 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.897+0000 7f309f7bc640 1 -- 172.21.15.87:0/232670861 wait complete. 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 Processor -- start 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 -- start start 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098102130 0x7f3098197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3098104a60 0x7f3098197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 0x7f309819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309810d330 con 0x7f3098107390 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309810d1b0 con 0x7f3098104a60 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f309810d4b0 con 0x7f3098102130 2024-03-21T08:56:34.896 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 0x7f309819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309d558640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098102130 0x7f3098197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 0x7f309819efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34900/0 (socket says 172.21.15.87:34900) 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309d558640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098102130 0x7f3098197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58720/0 (socket says 172.21.15.87:58720) 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 -- 172.21.15.87:0/671789324 learned_addr learned my addr 172.21.15.87:0/671789324 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309cd57640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3098104a60 0x7f3098197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 -- 172.21.15.87:0/671789324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098102130 msgr2=0x7f3098197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098102130 0x7f3098197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 -- 172.21.15.87:0/671789324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3098104a60 msgr2=0x7f3098197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3098104a60 0x7f3098197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 -- 172.21.15.87:0/671789324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f309819f650 con 0x7f3098107390 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309dd59640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 0x7f309819efe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f309000eb10 tx=0x7f309000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f308e7fc640 1 -- 172.21.15.87:0/671789324 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090004b20 con 0x7f3098107390 2024-03-21T08:56:34.897 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f308e7fc640 1 -- 172.21.15.87:0/671789324 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3090004cc0 con 0x7f3098107390 2024-03-21T08:56:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f309819f940 con 0x7f3098107390 2024-03-21T08:56:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.898+0000 7f308e7fc640 1 -- 172.21.15.87:0/671789324 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3090010760 con 0x7f3098107390 2024-03-21T08:56:34.898 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.899+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f309819feb0 con 0x7f3098107390 2024-03-21T08:56:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f308e7fc640 1 -- 172.21.15.87:0/671789324 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3090005020 con 0x7f3098107390 2024-03-21T08:56:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 Processor -- start 2024-03-21T08:56:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 -- start start 2024-03-21T08:56:34.904 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 0x56431ce61f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f308e7fc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30840a2fa0 0x7f30840a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce67040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce676b0 0x56431ce69b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431cdc2c20 con 0x56431ce676b0 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431cd6ba40 con 0x56431ce62540 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f30a239c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431cdbecc0 con 0x56431ce61b50 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f309dd59640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce676b0 0x56431ce69b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f309d558640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30840a2fa0 0x7f30840a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f309cd57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 0x56431ce61f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.901+0000 7f309cd57640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 0x56431ce61f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58736/0 (socket says 172.21.15.87:58736) 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 -- 172.21.15.87:0/3545649210 learned_addr learned my addr 172.21.15.87:0/3545649210 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce67040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f308e7fc640 1 -- 172.21.15.87:0/671789324 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f30900cf9d0 con 0x7f3098107390 2024-03-21T08:56:34.905 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 -- 172.21.15.87:0/3545649210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 msgr2=0x56431ce67040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce67040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 -- 172.21.15.87:0/3545649210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce676b0 msgr2=0x56431ce69b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce676b0 0x56431ce69b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 -- 172.21.15.87:0/3545649210 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431cd8df90 con 0x56431ce61b50 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309d558640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30840a2fa0 0x7f30840a5460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f30940095f0 tx=0x7f3094005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f309cd57640 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 0x56431ce61f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f308802cfd0 tx=0x7f308802bc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299e8551260baa67 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f30837fe640 1 -- 172.21.15.87:0/3545649210 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3088004030 con 0x56431ce61b50 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f30837fe640 1 -- 172.21.15.87:0/3545649210 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30880041d0 con 0x56431ce61b50 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.902+0000 7f30837fe640 1 -- 172.21.15.87:0/3545649210 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f308803f440 con 0x56431ce61b50 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 -- 172.21.15.87:0/3545649210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 msgr2=0x56431ce61f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 0x56431ce61f70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f308802cfd0 tx=0x7f308802bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:34.906 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 -- 172.21.15.87:0/3545649210 shutdown_connections 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce676b0 0x56431ce69b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce67040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 --2- 172.21.15.87:0/3545649210 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce61b50 0x56431ce61f70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 -- 172.21.15.87:0/3545649210 >> 172.21.15.87:0/3545649210 conn(0x56431ce25a60 msgr2=0x56431ce590b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 -- 172.21.15.87:0/3545649210 shutdown_connections 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 -- 172.21.15.87:0/3545649210 wait complete. 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 -- start start 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.903+0000 7f30a239c740 1 Processor -- start 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce6c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce6c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce676b0 0x56431ce73580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce6a600 con 0x56431ce61b50 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431cd7ee90 con 0x56431ce62540 2024-03-21T08:56:34.907 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce6a780 con 0x56431ce676b0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 10 client.0 _ll_register_callbacks cb 0x56431cad6ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce6c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce6c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34922/0 (socket says 172.21.15.87:34922) 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 -- 172.21.15.87:0/432512626 learned_addr learned my addr 172.21.15.87:0/432512626 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309dd59640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce676b0 0x56431ce73580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309cd57640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce6c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 -- 172.21.15.87:0/432512626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce676b0 msgr2=0x56431ce73580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce676b0 0x56431ce73580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 -- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 msgr2=0x56431ce6c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce6c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 -- 172.21.15.87:0/432512626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56431cd8b9d0 con 0x56431ce61b50 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f309d558640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce6c000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3094003970 tx=0x7f3094004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d302546b3352ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f3080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:34.908 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f3080ff9640 1 -- 172.21.15.87:0/432512626 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309400ecc0 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30880041d0 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f3080ff9640 1 -- 172.21.15.87:0/432512626 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f309401e420 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f3080ff9640 1 -- 172.21.15.87:0/432512626 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309402d070 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 10 client.15090 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.904+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56431cad72d0 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f3080ff9640 1 -- 172.21.15.87:0/432512626 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f30880041d0 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f3080ff9640 1 -- 172.21.15.87:0/432512626 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f3094026540 con 0x56431ce61b50 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f30a239c740 10 client.15090 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f30a239c740 10 client.15090 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f30a239c740 20 client.15090 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f30a239c740 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ce78880 0x56431ce77920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f30a239c740 4 client.15090 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:34.909 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 --> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56431cdd0160 con 0x56431ce78880 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f309cd57640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ce78880 0x56431ce77920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f3080ff9640 10 client.15090 ms_handle_connect on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.905+0000 7f309cd57640 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ce78880 0x56431ce77920 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6268e3c81ee2dd00 server_cookie=864b4245d4ad0de2 in_seq=0 out_seq=0 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.906+0000 7f3080ff9640 1 -- 172.21.15.87:0/432512626 <== mds.3 v2:172.21.15.100:6834/2548903221 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x56431cdd0160 con 0x56431ce78880 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.906+0000 7f3080ff9640 10 client.15090 handle_command_reply: tid=0 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.906+0000 7f30a239c740 1 client.15090 shutdown 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 msgr2=0x56431ce6c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce6c000 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3094003970 tx=0x7f3094004730 comp rx=0 tx=0).stop 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 shutdown_connections 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ce78880 0x56431ce77920 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce676b0 0x56431ce73580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce62540 0x56431ce6c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 --2- 172.21.15.87:0/432512626 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce6c000 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f3080ff9640 0 client.15090 ms_handle_reset on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:34.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 >> 172.21.15.87:0/432512626 conn(0x56431ce25a60 msgr2=0x56431ce582f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 shutdown_connections 2024-03-21T08:56:34.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 1 -- 172.21.15.87:0/432512626 wait complete. 2024-03-21T08:56:34.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.907+0000 7f30a239c740 20 client.15090 trim_cache size 0 max 16384 2024-03-21T08:56:34.924 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 Processor -- start 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 -- start start 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce63a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce63f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce73540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce780a0 con 0x56431ce61b50 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce78700 con 0x56431ce676b0 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f30a239c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce78d60 con 0x56431ce62540 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.923+0000 7f309d558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce63a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce63a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34932/0 (socket says 172.21.15.87:34932) 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 -- 172.21.15.87:0/3657807467 learned_addr learned my addr 172.21.15.87:0/3657807467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309cd57640 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce63f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309dd59640 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce73540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 -- 172.21.15.87:0/3657807467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 msgr2=0x56431ce63f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce63f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 -- 172.21.15.87:0/3657807467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 msgr2=0x56431ce73540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce73540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 -- 172.21.15.87:0/3657807467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56431cba16b0 con 0x56431ce61b50 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309dd59640 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce73540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f309d558640 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce63a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30940059f0 tx=0x7f3094003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1c7fa98b92599c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f3082ffd640 1 -- 172.21.15.87:0/3657807467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094040070 con 0x56431ce61b50 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f3082ffd640 1 -- 172.21.15.87:0/3657807467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3094004570 con 0x56431ce61b50 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.924+0000 7f3082ffd640 1 -- 172.21.15.87:0/3657807467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30940326e0 con 0x56431ce61b50 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- 172.21.15.87:0/3657807467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 msgr2=0x56431ce63a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce63a40 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f30940059f0 tx=0x7f3094003f10 comp rx=0 tx=0).stop 2024-03-21T08:56:34.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- 172.21.15.87:0/3657807467 shutdown_connections 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce73540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce63f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- 172.21.15.87:0/3657807467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce63a40 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- 172.21.15.87:0/3657807467 >> 172.21.15.87:0/3657807467 conn(0x56431ce25a60 msgr2=0x56431ce58bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- 172.21.15.87:0/3657807467 shutdown_connections 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- 172.21.15.87:0/3657807467 wait complete. 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- start start 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 Processor -- start 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce73380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce6fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce70360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce79a20 con 0x56431ce61b50 2024-03-21T08:56:34.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce793c0 con 0x56431ce676b0 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x56431ce7a080 con 0x56431ce62540 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f30a239c740 10 client.0 _ll_register_callbacks cb 0x7f3094004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f309d558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce73380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f309d558640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce73380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34934/0 (socket says 172.21.15.87:34934) 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f309d558640 1 -- 172.21.15.87:0/2830973636 learned_addr learned my addr 172.21.15.87:0/2830973636 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.925+0000 7f309cd57640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce6fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309dd59640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce70360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309d558640 1 -- 172.21.15.87:0/2830973636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 msgr2=0x56431ce6fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309d558640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce6fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309d558640 1 -- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 msgr2=0x56431ce70360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309d558640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce70360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309d558640 1 -- 172.21.15.87:0/2830973636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56431ced38e0 con 0x56431ce61b50 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f309d558640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce73380 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3094005c20 tx=0x7f3094026d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98782eb312a580ae server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:34.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f3080ff9640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f3080ff9640 1 -- 172.21.15.87:0/2830973636 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3094040070 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f3080ff9640 1 -- 172.21.15.87:0/2830973636 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f309400ecd0 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56431cdca4b0 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f3080ff9640 1 -- 172.21.15.87:0/2830973636 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f309404c050 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 10 client.15096 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56431ced4a10 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f3080ff9640 1 -- 172.21.15.87:0/2830973636 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x56431cdca4b0 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f3080ff9640 1 -- 172.21.15.87:0/2830973636 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f3094004ab0 con 0x56431ce61b50 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 10 client.15096 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 10 client.15096 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 20 client.15096 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.926+0000 7f30a239c740 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ced8ca0 0x56431ced4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 4 client.15096 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 --> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56431cd489a0 con 0x56431ced8ca0 2024-03-21T08:56:34.929 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f309cd57640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ced8ca0 0x56431ced4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f3080ff9640 10 client.15096 ms_handle_connect on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f309cd57640 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ced8ca0 0x56431ced4c40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8656bbfe3747a6c3 server_cookie=dfc6289b161283e in_seq=0 out_seq=0 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f3080ff9640 1 -- 172.21.15.87:0/2830973636 <== mds.3 v2:172.21.15.100:6834/2548903221 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56431cd489a0 con 0x56431ced8ca0 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f3080ff9640 10 client.15096 handle_command_reply: tid=0 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 client.15096 shutdown 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 msgr2=0x56431ce73380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce73380 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f3094005c20 tx=0x7f3094026d50 comp rx=0 tx=0).stop 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 shutdown_connections 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x56431ced8ca0 0x56431ced4c40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x56431ce676b0 0x56431ce70360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x56431ce62540 0x56431ce6fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.930 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f3080ff9640 0 client.15096 ms_handle_reset on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 --2- 172.21.15.87:0/2830973636 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x56431ce61b50 0x56431ce73380 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 >> 172.21.15.87:0/2830973636 conn(0x56431ce25a60 msgr2=0x56431ce5c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 shutdown_connections 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.927+0000 7f30a239c740 1 -- 172.21.15.87:0/2830973636 wait complete. 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f30a239c740 20 client.15096 trim_cache size 0 max 16384 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 3, 2024-03-21T08:56:34.931 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24425, 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 83.983910245999994, 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 114.199284741 2024-03-21T08:56:34.932 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30840a2fa0 msgr2=0x7f30840a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30840a2fa0 0x7f30840a5460 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f30940095f0 tx=0x7f3094005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 msgr2=0x7f309819efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 0x7f309819efe0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f309000eb10 tx=0x7f309000c2d0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.933 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 shutdown_connections 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f30840a2fa0 0x7f30840a5460 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3098107390 0x7f309819efe0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3098104a60 0x7f3098197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 --2- 172.21.15.87:0/671789324 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3098102130 0x7f3098197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 >> 172.21.15.87:0/671789324 conn(0x7f30980fbb40 msgr2=0x7f3098102bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 shutdown_connections 2024-03-21T08:56:34.934 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:34.928+0000 7f309f7bc640 1 -- 172.21.15.87:0/671789324 wait complete. 2024-03-21T08:56:34.935 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 -- 172.21.15.87:0/1175681497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d81021a0 msgr2=0x7f20d8104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 --2- 172.21.15.87:0/1175681497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d81021a0 0x7f20d8104590 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009860 tx=0x7f20c802bb70 comp rx=0 tx=0).stop 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 -- 172.21.15.87:0/1175681497 shutdown_connections 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 --2- 172.21.15.87:0/1175681497 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d8107400 0x7f20d81097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 --2- 172.21.15.87:0/1175681497 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d8104ad0 0x7f20d8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 --2- 172.21.15.87:0/1175681497 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d81021a0 0x7f20d8104590 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 -- 172.21.15.87:0/1175681497 >> 172.21.15.87:0/1175681497 conn(0x7f20d80fbb50 msgr2=0x7f20d80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 -- 172.21.15.87:0/1175681497 shutdown_connections 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.042+0000 7f20de463640 1 -- 172.21.15.87:0/1175681497 wait complete. 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 Processor -- start 2024-03-21T08:56:35.041 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 -- start start 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 0x7f20d8197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d8104ad0 0x7f20d8197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d8107400 0x7f20d819efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d810d340 con 0x7f20d81021a0 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d810d1c0 con 0x7f20d8107400 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20de463640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d810d4c0 con 0x7f20d8104ad0 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 0x7f20d8197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 0x7f20d8197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34956/0 (socket says 172.21.15.87:34956) 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.043+0000 7f20d7fff640 1 -- 172.21.15.87:0/1682275925 learned_addr learned my addr 172.21.15.87:0/1682275925 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d77fe640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d8104ad0 0x7f20d8197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.042 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20dca00640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d8107400 0x7f20d819efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d7fff640 1 -- 172.21.15.87:0/1682275925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d8104ad0 msgr2=0x7f20d8197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d7fff640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d8104ad0 0x7f20d8197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d7fff640 1 -- 172.21.15.87:0/1682275925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d8107400 msgr2=0x7f20d819efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d7fff640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d8107400 0x7f20d819efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d7fff640 1 -- 172.21.15.87:0/1682275925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d819f640 con 0x7f20d81021a0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d7fff640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 0x7f20d8197a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005940 tx=0x7f20c8032470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d57fa640 1 -- 172.21.15.87:0/1682275925 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8041020 con 0x7f20d81021a0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d57fa640 1 -- 172.21.15.87:0/1682275925 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c802c9f0 con 0x7f20d81021a0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d819f8d0 con 0x7f20d81021a0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20d57fa640 1 -- 172.21.15.87:0/1682275925 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c8039040 con 0x7f20d81021a0 2024-03-21T08:56:35.043 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.044+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d819fdc0 con 0x7f20d81021a0 2024-03-21T08:56:35.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.046+0000 7f20d57fa640 1 -- 172.21.15.87:0/1682275925 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f20c8044020 con 0x7f20d81021a0 2024-03-21T08:56:35.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 Processor -- start 2024-03-21T08:56:35.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 -- start start 2024-03-21T08:56:35.050 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d57fa640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f20c00a2f20 0x7f20c00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f61120740 0x556f61120b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f61125c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f611262a0 0x556f61128730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61085c20 con 0x556f61121130 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f6102ea40 con 0x556f611262a0 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20e1047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61081cc0 con 0x556f61120740 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20dca00640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f20c00a2f20 0x7f20c00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d57fa640 1 -- 172.21.15.87:0/1682275925 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f20c80f0d30 con 0x7f20d81021a0 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f61125c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f61125c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34972/0 (socket says 172.21.15.87:34972) 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f61120740 0x556f61120b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.051 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 -- 172.21.15.87:0/1553560730 learned_addr learned my addr 172.21.15.87:0/1553560730 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20dca00640 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f611262a0 0x556f61128730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 -- 172.21.15.87:0/1553560730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f61120740 msgr2=0x556f61120b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f61120740 0x556f61120b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 -- 172.21.15.87:0/1553560730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f611262a0 msgr2=0x556f61128730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f611262a0 0x556f61128730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20d77fe640 1 -- 172.21.15.87:0/1553560730 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f61050f90 con 0x556f61121130 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.047+0000 7f20dca00640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f20c00a2f20 0x7f20c00a53e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005fd0 tx=0x7f20d000e870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20d77fe640 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f61125c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009c60 tx=0x7f20cc011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a59f43f75fd3c179 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20be7fc640 1 -- 172.21.15.87:0/1553560730 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc004250 con 0x556f61121130 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20be7fc640 1 -- 172.21.15.87:0/1553560730 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20cc009e10 con 0x556f61121130 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20be7fc640 1 -- 172.21.15.87:0/1553560730 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc004d50 con 0x556f61121130 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 -- 172.21.15.87:0/1553560730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 msgr2=0x556f61125c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.052 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f61125c30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009c60 tx=0x7f20cc011810 comp rx=0 tx=0).stop 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 -- 172.21.15.87:0/1553560730 shutdown_connections 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f611262a0 0x556f61128730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f61125c30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 --2- 172.21.15.87:0/1553560730 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f61120740 0x556f61120b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 -- 172.21.15.87:0/1553560730 >> 172.21.15.87:0/1553560730 conn(0x556f610e8a60 msgr2=0x556f6111c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 -- 172.21.15.87:0/1553560730 shutdown_connections 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.048+0000 7f20e1047740 1 -- 172.21.15.87:0/1553560730 wait complete. 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 -- start start 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 Processor -- start 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61120740 0x556f6112abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f6112b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61132150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.053 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61129250 con 0x556f61121130 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61041e90 con 0x556f61120740 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f611293d0 con 0x556f611262a0 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f6112b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20e1047740 10 client.0 _ll_register_callbacks cb 0x556f60d99ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d77fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f6112b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34978/0 (socket says 172.21.15.87:34978) 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d77fe640 1 -- 172.21.15.87:0/3595714538 learned_addr learned my addr 172.21.15.87:0/3595714538 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20dca00640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61132150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d7fff640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61120740 0x556f6112abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d77fe640 1 -- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 msgr2=0x556f61132150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d77fe640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61132150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.049+0000 7f20d77fe640 1 -- 172.21.15.87:0/3595714538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61120740 msgr2=0x556f6112abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.054 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20d77fe640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61120740 0x556f6112abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20d77fe640 1 -- 172.21.15.87:0/3595714538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f6104e9d0 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20d77fe640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f6112b110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f20cc01b040 tx=0x7f20cc003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f5fa391425a8775 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20b77fe640 1 -- 172.21.15.87:0/3595714538 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc029070 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20b77fe640 1 -- 172.21.15.87:0/3595714538 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f20cc023430 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20cc009e10 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20b77fe640 1 -- 172.21.15.87:0/3595714538 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20cc035050 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 10 client.15108 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f60d9a2d0 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20b77fe640 1 -- 172.21.15.87:0/3595714538 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20cc009e10 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20b77fe640 1 -- 172.21.15.87:0/3595714538 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f20cc019430 con 0x556f61121130 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 10 client.15108 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 10 client.15108 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T08:56:35.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 20 client.15108 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f61137120 0x556f61136370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 4 client.15108 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.050+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 --> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f61093160 con 0x556f61137120 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.051+0000 7f20d7fff640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f61137120 0x556f61136370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.051+0000 7f20b77fe640 10 client.15108 ms_handle_connect on v2:172.21.15.105:6834/1031026542 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.051+0000 7f20d7fff640 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f61137120 0x556f61136370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=803ae342b35307cc server_cookie=f85cac7a37105bce in_seq=0 out_seq=0 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20b77fe640 1 -- 172.21.15.87:0/3595714538 <== mds.2 v2:172.21.15.105:6834/1031026542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x556f61093160 con 0x556f61137120 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20b77fe640 10 client.15108 handle_command_reply: tid=0 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 client.15108 shutdown 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 msgr2=0x556f6112b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f6112b110 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f20cc01b040 tx=0x7f20cc003820 comp rx=0 tx=0).stop 2024-03-21T08:56:35.056 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 shutdown_connections 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61132150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f61137120 0x556f61136370 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61121130 0x556f6112b110 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 --2- 172.21.15.87:0/3595714538 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61120740 0x556f6112abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 >> 172.21.15.87:0/3595714538 conn(0x556f610e8a60 msgr2=0x556f6111b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20b77fe640 0 client.15108 ms_handle_reset on v2:172.21.15.105:6834/1031026542 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 shutdown_connections 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 1 -- 172.21.15.87:0/3595714538 wait complete. 2024-03-21T08:56:35.057 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.052+0000 7f20e1047740 20 client.15108 trim_cache size 0 max 16384 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 Processor -- start 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 -- start start 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6111f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f61132110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f6111f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61136940 con 0x556f61120740 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61136fa0 con 0x556f61121130 2024-03-21T08:56:35.070 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20e1047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61137600 con 0x556f611262a0 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.068+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6111f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6111f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:34992/0 (socket says 172.21.15.87:34992) 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 -- 172.21.15.87:0/3640406774 learned_addr learned my addr 172.21.15.87:0/3640406774 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20dca00640 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f6111f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d77fe640 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f61132110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 -- 172.21.15.87:0/3640406774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 msgr2=0x556f6111f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f6111f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 -- 172.21.15.87:0/3640406774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 msgr2=0x556f61132110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f61132110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 -- 172.21.15.87:0/3640406774 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f60e646b0 con 0x556f61120740 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20d7fff640 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6111f2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20c80bd420 tx=0x7f20c802bb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18292bd20d8eb7f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20bdffb640 1 -- 172.21.15.87:0/3640406774 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80bdb90 con 0x556f61120740 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20bdffb640 1 -- 172.21.15.87:0/3640406774 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c80bd490 con 0x556f61120740 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.069+0000 7f20bdffb640 1 -- 172.21.15.87:0/3640406774 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80bb5c0 con 0x556f61120740 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- 172.21.15.87:0/3640406774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 msgr2=0x556f6111f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6111f2f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f20c80bd420 tx=0x7f20c802bb70 comp rx=0 tx=0).stop 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- 172.21.15.87:0/3640406774 shutdown_connections 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f6111f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f61132110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- 172.21.15.87:0/3640406774 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6111f2f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- 172.21.15.87:0/3640406774 >> 172.21.15.87:0/3640406774 conn(0x556f610e8a60 msgr2=0x556f61122530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- 172.21.15.87:0/3640406774 shutdown_connections 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- 172.21.15.87:0/3640406774 wait complete. 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- start start 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 Processor -- start 2024-03-21T08:56:35.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6112da80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f6112dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61130fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f611382c0 con 0x556f61120740 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61137c60 con 0x556f61121130 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f61138920 con 0x556f611262a0 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.070+0000 7f20e1047740 10 client.0 _ll_register_callbacks cb 0x7f20c8005ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6112da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6112da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35002/0 (socket says 172.21.15.87:35002) 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 -- 172.21.15.87:0/2815686937 learned_addr learned my addr 172.21.15.87:0/2815686937 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d77fe640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f6112dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20dca00640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61130fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 -- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 msgr2=0x556f61130fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61130fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 -- 172.21.15.87:0/2815686937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 msgr2=0x556f6112dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f6112dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 -- 172.21.15.87:0/2815686937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f611315d0 con 0x556f61120740 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d77fe640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f6112dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20d7fff640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6112da80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20c80b9e60 tx=0x7f20c8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a6d48488fd0e04 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20b77fe640 1 -- 172.21.15.87:0/2815686937 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c80c5ea0 con 0x556f61120740 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20b77fe640 1 -- 172.21.15.87:0/2815686937 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f20c8032640 con 0x556f61120740 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20c80bd490 con 0x556f61120740 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20b77fe640 1 -- 172.21.15.87:0/2815686937 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20c80bb5a0 con 0x556f61120740 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20e1047740 10 client.15114 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:35.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556f61192400 con 0x556f61120740 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20b77fe640 1 -- 172.21.15.87:0/2815686937 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f20c80bd490 con 0x556f61120740 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20b77fe640 1 -- 172.21.15.87:0/2815686937 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f20c803c030 con 0x556f61120740 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20e1047740 10 client.15114 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20e1047740 10 client.15114 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.071+0000 7f20e1047740 20 client.15114 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f611968a0 0x556f611925d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 4 client.15114 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 --> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] -- command(tid 0: {"prefix": "status"}) v1 -- 0x556f6108d4b0 con 0x556f611968a0 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20d77fe640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f611968a0 0x556f611925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20b77fe640 10 client.15114 ms_handle_connect on v2:172.21.15.105:6834/1031026542 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20d77fe640 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f611968a0 0x556f611925d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b9123c36d41857ed server_cookie=959e1dbd28eeb4a5 in_seq=0 out_seq=0 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20b77fe640 1 -- 172.21.15.87:0/2815686937 <== mds.2 v2:172.21.15.105:6834/1031026542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x556f6108d4b0 con 0x556f611968a0 2024-03-21T08:56:35.075 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20b77fe640 10 client.15114 handle_command_reply: tid=0 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 client.15114 shutdown 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 msgr2=0x556f6112da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6112da80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f20c80b9e60 tx=0x7f20c8005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 shutdown_connections 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x556f611262a0 0x556f61130fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x556f61121130 0x556f6112dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x556f611968a0 0x556f611925d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 --2- 172.21.15.87:0/2815686937 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x556f61120740 0x556f6112da80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20b77fe640 0 client.15114 ms_handle_reset on v2:172.21.15.105:6834/1031026542 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 >> 172.21.15.87:0/2815686937 conn(0x556f610e8a60 msgr2=0x556f6111aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 shutdown_connections 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.072+0000 7f20e1047740 1 -- 172.21.15.87:0/2815686937 wait complete. 2024-03-21T08:56:35.076 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20e1047740 20 client.15114 trim_cache size 0 max 16384 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 2, 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24451, 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 85.136542434999996, 2024-03-21T08:56:35.077 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:35.078 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:35.078 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:35.078 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 111.510192674 2024-03-21T08:56:35.078 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f20c00a2f20 msgr2=0x7f20c00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f20c00a2f20 0x7f20c00a53e0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f20d0005fd0 tx=0x7f20d000e870 comp rx=0 tx=0).stop 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 msgr2=0x7f20d8197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 0x7f20d8197a20 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005940 tx=0x7f20c8032470 comp rx=0 tx=0).stop 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 shutdown_connections 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f20c00a2f20 0x7f20c00a53e0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f20d8107400 0x7f20d819efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f20d8104ad0 0x7f20d8197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 --2- 172.21.15.87:0/1682275925 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f20d81021a0 0x7f20d8197a20 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.079 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 >> 172.21.15.87:0/1682275925 conn(0x7f20d80fbb50 msgr2=0x7f20d8102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 shutdown_connections 2024-03-21T08:56:35.080 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.073+0000 7f20de463640 1 -- 172.21.15.87:0/1682275925 wait complete. 2024-03-21T08:56:35.081 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T08:56:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 -- 172.21.15.87:0/266505516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 msgr2=0x7f1f5c106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/266505516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c106e50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1f58009900 tx=0x7f1f5802bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 -- 172.21.15.87:0/266505516 shutdown_connections 2024-03-21T08:56:35.194 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/266505516 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c107390 0x7f1f5c1097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/266505516 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c106e50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/266505516 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f5c102130 0x7f1f5c104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 -- 172.21.15.87:0/266505516 >> 172.21.15.87:0/266505516 conn(0x7f1f5c0fbb40 msgr2=0x7f1f5c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 -- 172.21.15.87:0/266505516 shutdown_connections 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 -- 172.21.15.87:0/266505516 wait complete. 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 Processor -- start 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.188+0000 7f1f63ff4640 1 -- start start 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c102130 0x7f1f5c197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f5c107390 0x7f1f5c19f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.195 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10d340 con 0x7f1f5c104a60 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10d1c0 con 0x7f1f5c102130 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5c10d4c0 con 0x7f1f5c107390 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35020/0 (socket says 172.21.15.87:35020) 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 -- 172.21.15.87:0/3758744176 learned_addr learned my addr 172.21.15.87:0/3758744176 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f61d90640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c102130 0x7f1f5c197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f62591640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f5c107390 0x7f1f5c19f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 -- 172.21.15.87:0/3758744176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f5c107390 msgr2=0x7f1f5c19f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f5c107390 0x7f1f5c19f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.196 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 -- 172.21.15.87:0/3758744176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c102130 msgr2=0x7f1f5c197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c102130 0x7f1f5c197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 -- 172.21.15.87:0/3758744176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f5c19f690 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f61d90640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c102130 0x7f1f5c197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f6158f640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c197fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f58000c00 tx=0x7f1f5802c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f4effd640 1 -- 172.21.15.87:0/3758744176 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58041070 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f5c19f920 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f4effd640 1 -- 172.21.15.87:0/3758744176 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f58004450 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f4effd640 1 -- 172.21.15.87:0/3758744176 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f58003df0 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.189+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f5c19fe60 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.191+0000 7f1f4effd640 1 -- 172.21.15.87:0/3758744176 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1f58044070 con 0x7f1f5c104a60 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 Processor -- start 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 -- start start 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e18b50 0x5563f8e18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.197 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e19540 0x5563f8e1e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e20b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8d79c20 con 0x5563f8e1e6b0 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8d22a40 con 0x5563f8e19540 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f66be3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8d75cc0 con 0x5563f8e18b50 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e20b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f61d90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e19540 0x5563f8e1e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f62591640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e18b50 0x5563f8e18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e20b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35026/0 (socket says 172.21.15.87:35026) 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 -- 172.21.15.87:0/1958639834 learned_addr learned my addr 172.21.15.87:0/1958639834 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.198 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f4effd640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f480a2f20 0x7f1f480a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f62591640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f480a2f20 0x7f1f480a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f4effd640 1 -- 172.21.15.87:0/3758744176 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1f580f0e50 con 0x7f1f5c104a60 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 -- 172.21.15.87:0/1958639834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e18b50 msgr2=0x5563f8e18f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e18b50 0x5563f8e18f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 -- 172.21.15.87:0/1958639834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e19540 msgr2=0x5563f8e1e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.192+0000 7f1f6158f640 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e19540 0x5563f8e1e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f6158f640 1 -- 172.21.15.87:0/1958639834 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563f8d44f90 con 0x5563f8e1e6b0 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f61d90640 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e19540 0x5563f8e1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f62591640 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e18b50 0x5563f8e18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.199 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f6158f640 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e20b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1f580c2880 tx=0x7f1f58003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c215bce1b9f35541 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f47fff640 1 -- 172.21.15.87:0/1958639834 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5804b070 con 0x5563f8e1e6b0 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f62591640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f480a2f20 0x7f1f480a53e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5563f8e19da0 tx=0x7f1f54004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f47fff640 1 -- 172.21.15.87:0/1958639834 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f580a8070 con 0x5563f8e1e6b0 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.193+0000 7f1f47fff640 1 -- 172.21.15.87:0/1958639834 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f580bd640 con 0x5563f8e1e6b0 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- 172.21.15.87:0/1958639834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 msgr2=0x5563f8e20b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e20b40 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1f580c2880 tx=0x7f1f58003f90 comp rx=0 tx=0).stop 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- 172.21.15.87:0/1958639834 shutdown_connections 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e20b40 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e19540 0x5563f8e1e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- 172.21.15.87:0/1958639834 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e18b50 0x5563f8e18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- 172.21.15.87:0/1958639834 >> 172.21.15.87:0/1958639834 conn(0x5563f8ddca60 msgr2=0x5563f8e144c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- 172.21.15.87:0/1958639834 shutdown_connections 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- 172.21.15.87:0/1958639834 wait complete. 2024-03-21T08:56:35.200 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- start start 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 Processor -- start 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e22e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 0x5563f8e23340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e1e6b0 0x5563f8e2a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e21420 con 0x5563f8e19540 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8d35e90 con 0x5563f8e18b50 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.194+0000 7f1f66be3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e215a0 con 0x5563f8e1e6b0 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 0x5563f8e23340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f66be3740 10 client.0 _ll_register_callbacks cb 0x5563f8a8dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 0x5563f8e23340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35036/0 (socket says 172.21.15.87:35036) 2024-03-21T08:56:35.201 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 -- 172.21.15.87:0/2791652436 learned_addr learned my addr 172.21.15.87:0/2791652436 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f62591640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e1e6b0 0x5563f8e2a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f61d90640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e22e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 -- 172.21.15.87:0/2791652436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e1e6b0 msgr2=0x5563f8e2a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e1e6b0 0x5563f8e2a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 -- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 msgr2=0x5563f8e22e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e22e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 -- 172.21.15.87:0/2791652436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563f8d429d0 con 0x5563f8e19540 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f61d90640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e22e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f6158f640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 0x5563f8e23340 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1f580bd340 tx=0x7f1f58005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1be03229c8a52fb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f457fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f457fa640 1 -- 172.21.15.87:0/2791652436 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5804b040 con 0x5563f8e19540 2024-03-21T08:56:35.202 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f457fa640 1 -- 172.21.15.87:0/2791652436 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1f580b9cb0 con 0x5563f8e19540 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5563f8d87160 con 0x5563f8e19540 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f457fa640 1 -- 172.21.15.87:0/2791652436 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f580bd410 con 0x5563f8e19540 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f66be3740 10 client.15126 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5563f8a8e2d0 con 0x5563f8e19540 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f457fa640 1 -- 172.21.15.87:0/2791652436 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x5563f8d87160 con 0x5563f8e19540 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f457fa640 1 -- 172.21.15.87:0/2791652436 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1f580a6bd0 con 0x5563f8e19540 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.195+0000 7f1f66be3740 10 client.15126 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f66be3740 10 client.15126 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f66be3740 20 client.15126 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f66be3740 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e2f360 0x5563f8e2e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f66be3740 4 client.15126 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:35.203 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 --> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563f8d814b0 con 0x5563f8e2f360 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f61d90640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e2f360 0x5563f8e2e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f61d90640 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e2f360 0x5563f8e2e4e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dc0229c2b4024d88 server_cookie=ef43e36ee70552a6 in_seq=0 out_seq=0 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.196+0000 7f1f457fa640 10 client.15126 ms_handle_connect on v2:172.21.15.100:6838/214101858 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f457fa640 1 -- 172.21.15.87:0/2791652436 <== mds.1 v2:172.21.15.100:6838/214101858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5563f8d814b0 con 0x5563f8e2f360 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f457fa640 10 client.15126 handle_command_reply: tid=0 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 client.15126 shutdown 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 msgr2=0x5563f8e23340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 0x5563f8e23340 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1f580bd340 tx=0x7f1f58005e50 comp rx=0 tx=0).stop 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 shutdown_connections 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e1e6b0 0x5563f8e2a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.204 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e2f360 0x5563f8e2e4e0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e19540 0x5563f8e23340 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f457fa640 0 client.15126 ms_handle_reset on v2:172.21.15.100:6838/214101858 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 --2- 172.21.15.87:0/2791652436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e22e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 >> 172.21.15.87:0/2791652436 conn(0x5563f8ddca60 msgr2=0x5563f8e0f2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 shutdown_connections 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 1 -- 172.21.15.87:0/2791652436 wait complete. 2024-03-21T08:56:35.205 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.197+0000 7f1f66be3740 20 client.15126 trim_cache size 0 max 16384 2024-03-21T08:56:35.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.213+0000 7f1f66be3740 1 Processor -- start 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 -- start start 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e14640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e2a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e0ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e2eb80 con 0x5563f8e1e6b0 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e2f1e0 con 0x5563f8e18b50 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f66be3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e2f840 con 0x5563f8e19540 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e0ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e0ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35044/0 (socket says 172.21.15.87:35044) 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 -- 172.21.15.87:0/104519824 learned_addr learned my addr 172.21.15.87:0/104519824 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f6158f640 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e2a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f61d90640 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e14640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 -- 172.21.15.87:0/104519824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 msgr2=0x5563f8e2a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e2a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 -- 172.21.15.87:0/104519824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 msgr2=0x5563f8e14640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e14640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 -- 172.21.15.87:0/104519824 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563f8b586b0 con 0x5563f8e1e6b0 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f61d90640 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e14640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f62591640 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e0ea60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005fd0 tx=0x7f1f54009b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=179734a731a748c5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f477fe640 1 -- 172.21.15.87:0/104519824 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54023be0 con 0x5563f8e1e6b0 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f477fe640 1 -- 172.21.15.87:0/104519824 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f54023d80 con 0x5563f8e1e6b0 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.214+0000 7f1f477fe640 1 -- 172.21.15.87:0/104519824 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5402b640 con 0x5563f8e1e6b0 2024-03-21T08:56:35.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 -- 172.21.15.87:0/104519824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 msgr2=0x5563f8e0ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e0ea60 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1f54005fd0 tx=0x7f1f54009b10 comp rx=0 tx=0).stop 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 -- 172.21.15.87:0/104519824 shutdown_connections 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e0ea60 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e2a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 --2- 172.21.15.87:0/104519824 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e14640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 -- 172.21.15.87:0/104519824 >> 172.21.15.87:0/104519824 conn(0x5563f8ddca60 msgr2=0x5563f8e13f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 -- 172.21.15.87:0/104519824 shutdown_connections 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 -- 172.21.15.87:0/104519824 wait complete. 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 -- start start 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.215+0000 7f1f66be3740 1 Processor -- start 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e26f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e27440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e8a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e30500 con 0x5563f8e1e6b0 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e2fea0 con 0x5563f8e18b50 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563f8e30b60 con 0x5563f8e19540 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 10 client.0 _ll_register_callbacks cb 0x7f1f50009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e8a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e8a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35052/0 (socket says 172.21.15.87:35052) 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f61d90640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e26f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 -- 172.21.15.87:0/3197260141 learned_addr learned my addr 172.21.15.87:0/3197260141 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f6158f640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e27440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 -- 172.21.15.87:0/3197260141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 msgr2=0x5563f8e27440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e27440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 -- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 msgr2=0x5563f8e26f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e26f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 -- 172.21.15.87:0/3197260141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5563f8e2a510 con 0x5563f8e1e6b0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f61d90640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f62591640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e8a3c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f54023830 tx=0x7f1f54023d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=defa43fb8525100e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f457fa640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f457fa640 1 -- 172.21.15.87:0/3197260141 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f54031040 con 0x5563f8e1e6b0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f457fa640 1 -- 172.21.15.87:0/3197260141 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1f54010a40 con 0x5563f8e1e6b0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1f54023d80 con 0x5563f8e1e6b0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f457fa640 1 -- 172.21.15.87:0/3197260141 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1f54043460 con 0x5563f8e1e6b0 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 10 client.15132 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:35.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.216+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5563f8e8b870 con 0x5563f8e1e6b0 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f457fa640 1 -- 172.21.15.87:0/3197260141 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1f54023d80 con 0x5563f8e1e6b0 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f457fa640 1 -- 172.21.15.87:0/3197260141 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f1f54034030 con 0x5563f8e1e6b0 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 10 client.15132 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 10 client.15132 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 20 client.15132 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e8fbc0 0x5563f8e8ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 4 client.15132 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 --> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5563f8cff9a0 con 0x5563f8e8fbc0 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f61d90640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e8fbc0 0x5563f8e8ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f457fa640 10 client.15132 ms_handle_connect on v2:172.21.15.100:6838/214101858 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f61d90640 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e8fbc0 0x5563f8e8ba40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=442135872ffeda81 server_cookie=ea6fd479fe119be0 in_seq=0 out_seq=0 2024-03-21T08:56:35.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f457fa640 1 -- 172.21.15.87:0/3197260141 <== mds.1 v2:172.21.15.100:6838/214101858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x5563f8cff9a0 con 0x5563f8e8fbc0 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f457fa640 10 client.15132 handle_command_reply: tid=0 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.217+0000 7f1f66be3740 1 client.15132 shutdown 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 msgr2=0x5563f8e8a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e8a3c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f1f54023830 tx=0x7f1f54023d10 comp rx=0 tx=0).stop 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 shutdown_connections 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x5563f8e8fbc0 0x5563f8e8ba40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5563f8e1e6b0 0x5563f8e8a3c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5563f8e19540 0x5563f8e27440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 --2- 172.21.15.87:0/3197260141 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5563f8e18b50 0x5563f8e26f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f457fa640 0 client.15132 ms_handle_reset on v2:172.21.15.100:6838/214101858 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 >> 172.21.15.87:0/3197260141 conn(0x5563f8ddca60 msgr2=0x5563f8e17700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 shutdown_connections 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 1 -- 172.21.15.87:0/3197260141 wait complete. 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f66be3740 20 client.15132 trim_cache size 0 max 16384 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 1, 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24482, 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:35.223 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 85.673948625999998, 2024-03-21T08:56:35.224 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:35.224 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:35.224 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:35.224 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 94.849109523999999 2024-03-21T08:56:35.224 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f480a2f20 msgr2=0x7f1f480a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f480a2f20 0x7f1f480a53e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5563f8e19da0 tx=0x7f1f54004ab0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 msgr2=0x7f1f5c197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c197fb0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1f58000c00 tx=0x7f1f5802c640 comp rx=0 tx=0).stop 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 shutdown_connections 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1f480a2f20 0x7f1f480a53e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1f5c107390 0x7f1f5c19f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1f5c104a60 0x7f1f5c197fb0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 --2- 172.21.15.87:0/3758744176 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1f5c102130 0x7f1f5c197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 >> 172.21.15.87:0/3758744176 conn(0x7f1f5c0fbb40 msgr2=0x7f1f5c1029d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 shutdown_connections 2024-03-21T08:56:35.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.218+0000 7f1f63ff4640 1 -- 172.21.15.87:0/3758744176 wait complete. 2024-03-21T08:56:35.226 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:34 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/310260423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.226 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:34 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2729180199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.226 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:34 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2776658931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.227 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-21T08:56:35.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:34 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/310260423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:34 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2729180199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:34 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2776658931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:34 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/310260423' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:34 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2729180199' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:34 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2776658931' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:35.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 -- 172.21.15.87:0/3107877143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0101790 msgr2=0x7f42f0101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 --2- 172.21.15.87:0/3107877143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0101790 0x7f42f0101c10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009ca0 tx=0x7f42e402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.331 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 -- 172.21.15.87:0/3107877143 shutdown_connections 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 --2- 172.21.15.87:0/3107877143 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f0102150 0x7f42f01089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 --2- 172.21.15.87:0/3107877143 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0101790 0x7f42f0101c10 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 --2- 172.21.15.87:0/3107877143 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f01005a0 0x7f42f01009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 -- 172.21.15.87:0/3107877143 >> 172.21.15.87:0/3107877143 conn(0x7f42f00fbb30 msgr2=0x7f42f00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 -- 172.21.15.87:0/3107877143 shutdown_connections 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.333+0000 7f42f5348640 1 -- 172.21.15.87:0/3107877143 wait complete. 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 Processor -- start 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 -- start start 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f01005a0 0x7f42f0069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0101790 0x7f42f019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 0x7f42f019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f010c230 con 0x7f42f0102150 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f010c0b0 con 0x7f42f01005a0 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42f5348640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f010c3b0 con 0x7f42f0101790 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 0x7f42f019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 0x7f42f019ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35060/0 (socket says 172.21.15.87:35060) 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42eeffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f01005a0 0x7f42f0069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 -- 172.21.15.87:0/3532641795 learned_addr learned my addr 172.21.15.87:0/3532641795 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0101790 0x7f42f019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 -- 172.21.15.87:0/3532641795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0101790 msgr2=0x7f42f019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0101790 0x7f42f019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.333 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 -- 172.21.15.87:0/3532641795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f01005a0 msgr2=0x7f42f0069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f01005a0 0x7f42f0069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42ef7fe640 1 -- 172.21.15.87:0/3532641795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f019f610 con 0x7f42f0102150 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.334+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f01005a0 0x7f42f0069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.335+0000 7f42ef7fe640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 0x7f42f019ed50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f42e000e970 tx=0x7f42e000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.335+0000 7f42d7fff640 1 -- 172.21.15.87:0/3532641795 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e000ccc0 con 0x7f42f0102150 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.335+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f019f900 con 0x7f42f0102150 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.335+0000 7f42d7fff640 1 -- 172.21.15.87:0/3532641795 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e000ce60 con 0x7f42f0102150 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.335+0000 7f42d7fff640 1 -- 172.21.15.87:0/3532641795 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0010490 con 0x7f42f0102150 2024-03-21T08:56:35.334 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.335+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f019fe20 con 0x7f42f0102150 2024-03-21T08:56:35.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42d7fff640 1 -- 172.21.15.87:0/3532641795 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f42e0005020 con 0x7f42f0102150 2024-03-21T08:56:35.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 Processor -- start 2024-03-21T08:56:35.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 -- start start 2024-03-21T08:56:35.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 0x55f27e673b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e678c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e67b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e5d8c20 con 0x55f27e673740 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e581a40 con 0x55f27e674130 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42f800c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e5d4cc0 con 0x55f27e6792a0 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42d7fff640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42d80a2f20 0x7f42d80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 0x55f27e673b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 0x55f27e673b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35070/0 (socket says 172.21.15.87:35070) 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.337+0000 7f42ee7fc640 1 -- 172.21.15.87:0/234698453 learned_addr learned my addr 172.21.15.87:0/234698453 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42eeffd640 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e678c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e67b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ef7fe640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42d80a2f20 0x7f42d80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.343 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 -- 172.21.15.87:0/234698453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 msgr2=0x55f27e67b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e67b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42d7fff640 1 -- 172.21.15.87:0/3532641795 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f42e00cf7f0 con 0x7f42f0102150 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 -- 172.21.15.87:0/234698453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 msgr2=0x55f27e678c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e678c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 -- 172.21.15.87:0/234698453 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f27e5a3f90 con 0x55f27e673740 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42eeffd640 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e678c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 0x55f27e673b60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009430 tx=0x7f42e4000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134a7da031d185b0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42ef7fe640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42d80a2f20 0x7f42d80a53e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42e00cd370 tx=0x7f42e00c6c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42d57fa640 1 -- 172.21.15.87:0/234698453 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4039040 con 0x55f27e673740 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42d57fa640 1 -- 172.21.15.87:0/234698453 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e4003970 con 0x55f27e673740 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.338+0000 7f42d57fa640 1 -- 172.21.15.87:0/234698453 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4004330 con 0x55f27e673740 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- 172.21.15.87:0/234698453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 msgr2=0x55f27e673b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 0x55f27e673b60 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009430 tx=0x7f42e4000f40 comp rx=0 tx=0).stop 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- 172.21.15.87:0/234698453 shutdown_connections 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e67b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e678c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- 172.21.15.87:0/234698453 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e673740 0x55f27e673b60 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- 172.21.15.87:0/234698453 >> 172.21.15.87:0/234698453 conn(0x55f27e63ba60 msgr2=0x55f27e66f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- 172.21.15.87:0/234698453 shutdown_connections 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- 172.21.15.87:0/234698453 wait complete. 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- start start 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 Processor -- start 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e67dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e67e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e6792a0 0x55f27e685200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e67c1f0 con 0x55f27e6792a0 2024-03-21T08:56:35.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e594e90 con 0x55f27e674130 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e67c370 con 0x55f27e673740 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42f800c740 10 client.0 _ll_register_callbacks cb 0x55f27e2ecac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e6792a0 0x55f27e685200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.339+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e67dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e67dc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58938/0 (socket says 172.21.15.87:58938) 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 -- 172.21.15.87:0/805030803 learned_addr learned my addr 172.21.15.87:0/805030803 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42eeffd640 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e67e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 -- 172.21.15.87:0/805030803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 msgr2=0x55f27e67e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e67e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.346 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 -- 172.21.15.87:0/805030803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e6792a0 msgr2=0x55f27e685200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e6792a0 0x55f27e685200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 -- 172.21.15.87:0/805030803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f27e5a19d0 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e67dc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42e402c700 tx=0x7f42e402c0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1038b6dfb625a8f server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.105:3300/0 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42d2ff5640 1 -- 172.21.15.87:0/805030803 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4039040 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42d2ff5640 1 -- 172.21.15.87:0/805030803 <== mon.1 v2:172.21.15.105:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f42e4032960 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42e4003970 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42d2ff5640 1 -- 172.21.15.87:0/805030803 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e4004110 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42f800c740 10 client.24967 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f27e2ed2d0 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.340+0000 7f42d2ff5640 1 -- 172.21.15.87:0/805030803 <== mon.1 v2:172.21.15.105:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e4003970 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42d2ff5640 1 -- 172.21.15.87:0/805030803 <== mon.1 v2:172.21.15.105:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f42e4050040 con 0x55f27e673740 2024-03-21T08:56:35.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42f800c740 10 client.24967 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42f800c740 10 client.24967 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42f800c740 20 client.24967 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42f800c740 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e689e20 0x55f27e68e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42f800c740 4 client.24967 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 --> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f27e5e6160 con 0x55f27e689e20 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42eeffd640 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e689e20 0x55f27e68e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42eeffd640 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e689e20 0x55f27e68e2a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275d20690881c507 server_cookie=91167319ed65f3e2 in_seq=0 out_seq=0 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.341+0000 7f42d2ff5640 10 client.24967 ms_handle_connect on v2:172.21.15.105:6838/892092628 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42d2ff5640 1 -- 172.21.15.87:0/805030803 <== mds.0 v2:172.21.15.105:6838/892092628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55f27e5e6160 con 0x55f27e689e20 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42d2ff5640 10 client.24967 handle_command_reply: tid=0 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 client.24967 shutdown 2024-03-21T08:56:35.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 msgr2=0x55f27e67dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e67dc80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f42e402c700 tx=0x7f42e402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 shutdown_connections 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e689e20 0x55f27e68e2a0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e6792a0 0x55f27e685200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e674130 0x55f27e67e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42d2ff5640 0 client.24967 ms_handle_reset on v2:172.21.15.105:6838/892092628 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 --2- 172.21.15.87:0/805030803 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e67dc80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 >> 172.21.15.87:0/805030803 conn(0x55f27e63ba60 msgr2=0x55f27e66e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 shutdown_connections 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 1 -- 172.21.15.87:0/805030803 wait complete. 2024-03-21T08:56:35.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.342+0000 7f42f800c740 20 client.24967 trim_cache size 0 max 16384 2024-03-21T08:56:35.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.358+0000 7f42f800c740 1 Processor -- start 2024-03-21T08:56:35.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.358+0000 7f42f800c740 1 -- start start 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42f800c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e673740 0x55f27e6932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42f800c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6851c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42f800c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e6722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42f800c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e689640 con 0x55f27e674130 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42f800c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e689ca0 con 0x55f27e673740 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42f800c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e68a300 con 0x55f27e6792a0 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6851c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6851c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35088/0 (socket says 172.21.15.87:35088) 2024-03-21T08:56:35.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 -- 172.21.15.87:0/3124314130 learned_addr learned my addr 172.21.15.87:0/3124314130 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e673740 0x55f27e6932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e6722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 -- 172.21.15.87:0/3124314130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 msgr2=0x55f27e6722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e6722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 -- 172.21.15.87:0/3124314130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e673740 msgr2=0x55f27e6932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e673740 0x55f27e6932d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 -- 172.21.15.87:0/3124314130 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f27e3b76b0 con 0x55f27e674130 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e6722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6851c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42e8005d00 tx=0x7f42e8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3176118bb71c2adb server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42d4ff9640 1 -- 172.21.15.87:0/3124314130 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e801f230 con 0x55f27e674130 2024-03-21T08:56:35.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42d4ff9640 1 -- 172.21.15.87:0/3124314130 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e8010960 con 0x55f27e674130 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.359+0000 7f42d4ff9640 1 -- 172.21.15.87:0/3124314130 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e801fa60 con 0x55f27e674130 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- 172.21.15.87:0/3124314130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 msgr2=0x55f27e6851c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6851c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f42e8005d00 tx=0x7f42e8005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- 172.21.15.87:0/3124314130 shutdown_connections 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e6792a0 0x55f27e6722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6851c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- 172.21.15.87:0/3124314130 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e673740 0x55f27e6932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- 172.21.15.87:0/3124314130 >> 172.21.15.87:0/3124314130 conn(0x55f27e63ba60 msgr2=0x55f27e67c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- 172.21.15.87:0/3124314130 shutdown_connections 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- 172.21.15.87:0/3124314130 wait complete. 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- start start 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 Processor -- start 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e683360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.364 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6838a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e6792a0 0x55f27e683ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e68afc0 con 0x55f27e674130 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e68a960 con 0x55f27e6792a0 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f27e68b620 con 0x55f27e673740 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42f800c740 10 client.0 _ll_register_callbacks cb 0x7f42e8009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42eeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6838a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42eeffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6838a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:35102/0 (socket says 172.21.15.87:35102) 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42eeffd640 1 -- 172.21.15.87:0/3752641300 learned_addr learned my addr 172.21.15.87:0/3752641300 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e683360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.360+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e6792a0 0x55f27e683ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42eeffd640 1 -- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 msgr2=0x55f27e683360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.365 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e683360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42eeffd640 1 -- 172.21.15.87:0/3752641300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e6792a0 msgr2=0x55f27e683ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e6792a0 0x55f27e683ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42eeffd640 1 -- 172.21.15.87:0/3752641300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f27e684700 con 0x55f27e674130 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42eeffd640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6838a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f42e80126a0 tx=0x7f42e801f0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14ed4adb79e85a5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42d2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42d2ff5640 1 -- 172.21.15.87:0/3752641300 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e802d070 con 0x55f27e674130 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42d2ff5640 1 -- 172.21.15.87:0/3752641300 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f42e8004050 con 0x55f27e674130 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42e8010960 con 0x55f27e674130 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42d2ff5640 1 -- 172.21.15.87:0/3752641300 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42e8004250 con 0x55f27e674130 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42f800c740 10 client.15150 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:35.366 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55f27e6e56a0 con 0x55f27e674130 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42d2ff5640 1 -- 172.21.15.87:0/3752641300 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f42e8010960 con 0x55f27e674130 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42d2ff5640 1 -- 172.21.15.87:0/3752641300 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f42e8003920 con 0x55f27e674130 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42f800c740 10 client.15150 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42f800c740 10 client.15150 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.361+0000 7f42f800c740 20 client.15150 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e6e99f0 0x55f27e6e5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 4 client.15150 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 --> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55f27e5e04b0 con 0x55f27e6e99f0 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e6e99f0 0x55f27e6e5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42d2ff5640 10 client.15150 ms_handle_connect on v2:172.21.15.105:6838/892092628 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42ee7fc640 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e6e99f0 0x55f27e6e5980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38fed1b9defb8b87 server_cookie=5916ee5a7c9a97a4 in_seq=0 out_seq=0 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42d2ff5640 1 -- 172.21.15.87:0/3752641300 <== mds.0 v2:172.21.15.105:6838/892092628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55f27e5e04b0 con 0x55f27e6e99f0 2024-03-21T08:56:35.367 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42d2ff5640 10 client.15150 handle_command_reply: tid=0 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 client.15150 shutdown 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 msgr2=0x55f27e6838a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6838a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f42e80126a0 tx=0x7f42e801f0c0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 shutdown_connections 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x55f27e6e99f0 0x55f27e6e5980 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55f27e6792a0 0x55f27e683ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55f27e674130 0x55f27e6838a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42d2ff5640 0 client.15150 ms_handle_reset on v2:172.21.15.105:6838/892092628 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 --2- 172.21.15.87:0/3752641300 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55f27e673740 0x55f27e683360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 >> 172.21.15.87:0/3752641300 conn(0x55f27e63ba60 msgr2=0x55f27e6722f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 shutdown_connections 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.362+0000 7f42f800c740 1 -- 172.21.15.87:0/3752641300 wait complete. 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f800c740 20 client.15150 trim_cache size 0 max 16384 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24487, 2024-03-21T08:56:35.369 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 87.842020997000006, 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 103.540544748 2024-03-21T08:56:35.370 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42d80a2f20 msgr2=0x7f42d80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42d80a2f20 0x7f42d80a53e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f42e00cd370 tx=0x7f42e00c6c10 comp rx=0 tx=0).stop 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 msgr2=0x7f42f019ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 0x7f42f019ed50 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f42e000e970 tx=0x7f42e000ed70 comp rx=0 tx=0).stop 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 shutdown_connections 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f42d80a2f20 0x7f42d80a53e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f42f0102150 0x7f42f019ed50 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f42f0101790 0x7f42f019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 --2- 172.21.15.87:0/3532641795 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f42f01005a0 0x7f42f0069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:35.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 >> 172.21.15.87:0/3532641795 conn(0x7f42f00fbb30 msgr2=0x7f42f0104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 shutdown_connections 2024-03-21T08:56:35.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:35.363+0000 7f42f5348640 1 -- 172.21.15.87:0/3532641795 wait complete. 2024-03-21T08:56:35.373 INFO:tasks.cephadm:Setup complete, yielding 2024-03-21T08:56:35.373 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T08:56:35.387 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-03-21T08:56:35.387 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph orch status' 2024-03-21T08:56:35.630 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:36.026 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:35 smithi087 ceph-mon[32385]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:36.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:35 smithi105 ceph-mon[35993]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:36.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:35 smithi100 ceph-mon[36306]: pgmap v336: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 -- 172.21.15.87:0/1092017975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6940074bb0 msgr2=0x7f6940075010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 --2- 172.21.15.87:0/1092017975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6940074bb0 0x7f6940075010 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009ca0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 -- 172.21.15.87:0/1092017975 shutdown_connections 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 --2- 172.21.15.87:0/1092017975 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f69400745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 --2- 172.21.15.87:0/1092017975 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f694010a0a0 0x7f694010c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 --2- 172.21.15.87:0/1092017975 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6940074bb0 0x7f6940075010 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 -- 172.21.15.87:0/1092017975 >> 172.21.15.87:0/1092017975 conn(0x7f69400fd7a0 msgr2=0x7f69400ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 -- 172.21.15.87:0/1092017975 shutdown_connections 2024-03-21T08:56:37.345 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.348+0000 7f6944ba8700 1 -- 172.21.15.87:0/1092017975 wait complete. 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 Processor -- start 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 -- start start 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 0x7f694019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f694019aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f694010a0a0 0x7f69401a1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694010ffc0 con 0x7f694010a0a0 2024-03-21T08:56:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694010fe40 con 0x7f6940076180 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f6944ba8700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6940110140 con 0x7f6940074bb0 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f693ed9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f694010a0a0 0x7f69401a1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f693e59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 0x7f694019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f693dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f694019aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f693e59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 0x7f694019a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:58978/0 (socket says 172.21.15.87:58978) 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f693dd9b700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f694019aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:41148/0 (socket says 172.21.15.87:41148) 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.349+0000 7f693e59c700 1 -- 172.21.15.87:0/138762089 learned_addr learned my addr 172.21.15.87:0/138762089 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693e59c700 1 -- 172.21.15.87:0/138762089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 msgr2=0x7f694019aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693e59c700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f694019aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693e59c700 1 -- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f694010a0a0 msgr2=0x7f69401a1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693e59c700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f694010a0a0 0x7f69401a1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693e59c700 1 -- 172.21.15.87:0/138762089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69401a21e0 con 0x7f6940074bb0 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693dd9b700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f694019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693ed9d700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f694010a0a0 0x7f69401a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:37.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f693e59c700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 0x7f694019a580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f693000e9a0 tx=0x7f693000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:37.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930004220 con 0x7f6940074bb0 2024-03-21T08:56:37.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f69300043c0 con 0x7f6940074bb0 2024-03-21T08:56:37.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69401a24d0 con 0x7f6940074bb0 2024-03-21T08:56:37.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930003df0 con 0x7f6940074bb0 2024-03-21T08:56:37.349 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.350+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69401a2a60 con 0x7f6940074bb0 2024-03-21T08:56:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.352+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6940068e50 con 0x7f6940074bb0 2024-03-21T08:56:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.352+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f693000c810 con 0x7f6940074bb0 2024-03-21T08:56:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.353+0000 7f692f7fe700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f69280a3030 0x7f69280a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.353+0000 7f693dd9b700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f69280a3030 0x7f69280a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.353+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f69300cf510 con 0x7f6940074bb0 2024-03-21T08:56:37.350 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.353+0000 7f693dd9b700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f69280a3030 0x7f69280a54f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f693400b2e0 tx=0x7f693400b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:37.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.357+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f693009c370 con 0x7f6940074bb0 2024-03-21T08:56:37.570 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.574+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69400625d0 con 0x7f69280a3030 2024-03-21T08:56:37.572 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.576+0000 7f692f7fe700 1 -- 172.21.15.87:0/138762089 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f69400625d0 con 0x7f69280a3030 2024-03-21T08:56:37.572 INFO:teuthology.orchestra.run.smithi087.stdout:Backend: cephadm 2024-03-21T08:56:37.572 INFO:teuthology.orchestra.run.smithi087.stdout:Available: Yes 2024-03-21T08:56:37.572 INFO:teuthology.orchestra.run.smithi087.stdout:Paused: No 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f69280a3030 msgr2=0x7f69280a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f69280a3030 0x7f69280a54f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f693400b2e0 tx=0x7f693400b880 comp rx=0 tx=0).stop 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 msgr2=0x7f694019a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 0x7f694019a580 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f693000e9a0 tx=0x7f693000ecb0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 shutdown_connections 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f69280a3030 0x7f69280a54f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f694010a0a0 0x7f69401a1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6940074bb0 0x7f694019a580 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 --2- 172.21.15.87:0/138762089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6940076180 0x7f694019aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 >> 172.21.15.87:0/138762089 conn(0x7f69400fd7a0 msgr2=0x7f6940105ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 shutdown_connections 2024-03-21T08:56:37.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:37.578+0000 7f6944ba8700 1 -- 172.21.15.87:0/138762089 wait complete. 2024-03-21T08:56:38.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:37 smithi105 ceph-mon[35993]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T08:56:38.251 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:37 smithi087 ceph-mon[32385]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T08:56:38.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:37 smithi100 ceph-mon[36306]: pgmap v337: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 426 B/s wr, 9 op/s 2024-03-21T08:56:38.280 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ps' 2024-03-21T08:56:38.523 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:39.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:38 smithi105 ceph-mon[35993]: from='client.24979 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:39.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:38 smithi100 ceph-mon[36306]: from='client.24979 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:39.327 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:38 smithi087 ceph-mon[32385]: from='client.24979 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:40.030 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:39 smithi087 ceph-mon[32385]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:40.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:39 smithi105 ceph-mon[35993]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:40.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/3080700436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 msgr2=0x7f5c9c10a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:40.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/3080700436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 0x7f5c9c10a390 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f5c9400b0a0 tx=0x7f5c9400b3b0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.266 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/3080700436 shutdown_connections 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/3080700436 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9c102d40 0x7f5c9c105130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/3080700436 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 0x7f5c9c10a390 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/3080700436 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c105670 0x7f5c9c107a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/3080700436 >> 172.21.15.87:0/3080700436 conn(0x7f5c9c0fc6f0 msgr2=0x7f5c9c0feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/3080700436 shutdown_connections 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.255+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/3080700436 wait complete. 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 Processor -- start 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 -- start start 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 0x7f5c9c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9c105670 0x7f5c9c1988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 0x7f5c9c19f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c10dda0 con 0x7f5c9c107fa0 2024-03-21T08:56:40.267 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c10dc20 con 0x7f5c9c105670 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5ca0fbf700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c10df20 con 0x7f5c9c102d40 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5c9a59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 0x7f5c9c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5c9a59c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 0x7f5c9c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:60512/0 (socket says 172.21.15.87:60512) 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5c9a59c700 1 -- 172.21.15.87:0/457358006 learned_addr learned my addr 172.21.15.87:0/457358006 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.256+0000 7f5c9ad9d700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 0x7f5c9c19f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c99d9b700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9c105670 0x7f5c9c1988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c9a59c700 1 -- 172.21.15.87:0/457358006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9c105670 msgr2=0x7f5c9c1988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c9a59c700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9c105670 0x7f5c9c1988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c9a59c700 1 -- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 msgr2=0x7f5c9c19f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:40.268 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c9a59c700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 0x7f5c9c19f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c9a59c700 1 -- 172.21.15.87:0/457358006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c1a0010 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c9a59c700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 0x7f5c9c1983b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000e9d0 tx=0x7f5c9000ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c9000cbd0 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5c9000ee70 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9c1a0300 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90018660 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.257+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9c1a0840 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.258+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c9c068e50 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.259+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5c90010430 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.260+0000 7f5c8b7fe700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5c840a2fb0 0x7f5c840a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.260+0000 7f5c99d9b700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5c840a2fb0 0x7f5c840a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.260+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5c900cf7d0 con 0x7f5c9c102d40 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.260+0000 7f5c99d9b700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5c840a2fb0 0x7f5c840a5470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009c60 tx=0x7f5c8c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:40.269 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.264+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5c9009c630 con 0x7f5c9c102d40 2024-03-21T08:56:40.270 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:39 smithi100 ceph-mon[36306]: pgmap v338: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:40.464 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.468+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c9c0625d0 con 0x7f5c840a2fb0 2024-03-21T08:56:40.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.482+0000 7f5c8b7fe700 1 -- 172.21.15.87:0/457358006 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f5c9c0625d0 con 0x7f5c840a2fb0 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.a smithi087 running (103s) 16s ago 102s 18.7M - 19.0.0-2231-g2edfee68 237e43c7a65f d1d898c1ca91 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.b smithi100 running (100s) 33s ago 100s 20.0M - 19.0.0-2231-g2edfee68 237e43c7a65f 32d8de4201f7 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.c smithi105 running (114s) 33s ago 114s 17.0M - 19.0.0-2231-g2edfee68 237e43c7a65f bc8bc523e9fb 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.d smithi087 running (2m) 16s ago 2m 13.8M - 19.0.0-2231-g2edfee68 237e43c7a65f 11bdb4052c12 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.e smithi100 running (2m) 33s ago 119s 20.1M - 19.0.0-2231-g2edfee68 237e43c7a65f ae2d741befc5 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.f smithi105 running (117s) 33s ago 116s 20.5M - 19.0.0-2231-g2edfee68 237e43c7a65f 239b5b7b71bb 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.g smithi087 running (106s) 16s ago 105s 16.4M - 19.0.0-2231-g2edfee68 237e43c7a65f d27be41e4d22 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.h smithi100 running (111s) 33s ago 111s 18.4M - 19.0.0-2231-g2edfee68 237e43c7a65f fff9dbdc6e87 2024-03-21T08:56:40.478 INFO:teuthology.orchestra.run.smithi087.stdout:mds.i smithi105 running (109s) 33s ago 108s 26.2M - 19.0.0-2231-g2edfee68 237e43c7a65f 5c4370e9bdfe 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mds.j smithi087 running (2m) 16s ago 2m 16.2M - 19.0.0-2231-g2edfee68 237e43c7a65f b7c8e830bb67 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mds.k smithi100 running (2m) 33s ago 2m 18.3M - 19.0.0-2231-g2edfee68 237e43c7a65f 69d3153453ca 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mds.l smithi105 running (2m) 33s ago 2m 17.7M - 19.0.0-2231-g2edfee68 237e43c7a65f fa46257e6b43 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mgr.x smithi087 *:9283,8765 running (9m) 16s ago 9m 513M - 19.0.0-2231-g2edfee68 237e43c7a65f 9d5a6cfb7056 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mgr.y smithi100 *:8443,8765 running (7m) 33s ago 7m 451M - 19.0.0-2231-g2edfee68 237e43c7a65f 5457a075ab1d 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mgr.z smithi105 *:8443,8765 running (7m) 33s ago 7m 452M - 19.0.0-2231-g2edfee68 237e43c7a65f fbd2b271f54d 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mon.a smithi087 running (9m) 16s ago 9m 50.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f b1c901b5acae 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mon.b smithi100 running (7m) 33s ago 7m 40.0M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f 37d20ded27fc 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:mon.c smithi105 running (7m) 33s ago 7m 39.6M 2048M 19.0.0-2231-g2edfee68 237e43c7a65f c3d2d1514f72 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:osd.0 smithi087 running (7m) 16s ago 7m 48.5M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 0b5e0a8f4859 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:osd.1 smithi100 running (6m) 33s ago 6m 46.4M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 552a310e5bd8 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:osd.2 smithi105 running (6m) 33s ago 6m 59.4M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 2ab235da0c6d 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:osd.3 smithi087 running (5m) 16s ago 5m 43.6M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 1f1819d7bedb 2024-03-21T08:56:40.479 INFO:teuthology.orchestra.run.smithi087.stdout:osd.4 smithi100 running (5m) 33s ago 5m 52.9M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 315ad875c64e 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.5 smithi105 running (5m) 33s ago 5m 53.8M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 6d007d968483 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.6 smithi087 running (4m) 16s ago 4m 51.4M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f 5414e0ba0aad 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.7 smithi100 running (4m) 33s ago 4m 44.7M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 7a71526d3540 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.8 smithi105 running (3m) 33s ago 3m 43.2M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 4df4d0b24529 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.9 smithi087 running (3m) 16s ago 3m 54.3M 4272M 19.0.0-2231-g2edfee68 237e43c7a65f f373877be474 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.10 smithi100 running (2m) 33s ago 2m 45.1M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 2d7846dffb60 2024-03-21T08:56:40.480 INFO:teuthology.orchestra.run.smithi087.stdout:osd.11 smithi105 running (2m) 33s ago 2m 40.4M 4270M 19.0.0-2231-g2edfee68 237e43c7a65f 5cd67200ce2f 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5c840a2fb0 msgr2=0x7f5c840a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5c840a2fb0 0x7f5c840a5470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5c8c009c60 tx=0x7f5c8c009260 comp rx=0 tx=0).stop 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 msgr2=0x7f5c9c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 0x7f5c9c1983b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000e9d0 tx=0x7f5c9000ece0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 shutdown_connections 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5c840a2fb0 0x7f5c840a5470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5c9c107fa0 0x7f5c9c19f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5c9c102d40 0x7f5c9c1983b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.481 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 --2- 172.21.15.87:0/457358006 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5c9c105670 0x7f5c9c1988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:40.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 >> 172.21.15.87:0/457358006 conn(0x7f5c9c0fc6f0 msgr2=0x7f5c9c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:40.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 shutdown_connections 2024-03-21T08:56:40.482 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:40.484+0000 7f5ca0fbf700 1 -- 172.21.15.87:0/457358006 wait complete. 2024-03-21T08:56:41.292 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph orch ls' 2024-03-21T08:56:41.532 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:42.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:41 smithi105 ceph-mon[35993]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:42.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:41 smithi105 ceph-mon[35993]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:42.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:41 smithi100 ceph-mon[36306]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:42.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:41 smithi100 ceph-mon[36306]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:42.381 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:41 smithi087 ceph-mon[32385]: from='client.24985 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:42.381 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:41 smithi087 ceph-mon[32385]: pgmap v339: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 426 B/s wr, 11 op/s 2024-03-21T08:56:43.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 -- 172.21.15.87:0/3269140846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 msgr2=0x7fd5c410a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3269140846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 0x7fd5c410a6b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00b0a0 tx=0x7fd5bc00b3b0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 -- 172.21.15.87:0/3269140846 shutdown_connections 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3269140846 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3269140846 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 0x7fd5c410a6b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3269140846 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 0x7fd5c41038c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 -- 172.21.15.87:0/3269140846 >> 172.21.15.87:0/3269140846 conn(0x7fd5c40fbe00 msgr2=0x7fd5c40fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 -- 172.21.15.87:0/3269140846 shutdown_connections 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.376+0000 7fd5ca330700 1 -- 172.21.15.87:0/3269140846 wait complete. 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 Processor -- start 2024-03-21T08:56:43.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 -- start start 2024-03-21T08:56:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 0x7fd5c4198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 0x7fd5c419f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c410dd80 con 0x7fd5c4103e00 2024-03-21T08:56:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c410dc00 con 0x7fd5c4100480 2024-03-21T08:56:43.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5ca330700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c410df00 con 0x7fd5c4103460 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c37fe700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 0x7fd5c4198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c88cd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 0x7fd5c419f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c3fff700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c4198340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:46986/0 (socket says 172.21.15.87:46986) 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c3fff700 1 -- 172.21.15.87:0/3622204231 learned_addr learned my addr 172.21.15.87:0/3622204231 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c37fe700 1 -- 172.21.15.87:0/3622204231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 msgr2=0x7fd5c4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c37fe700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c37fe700 1 -- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 msgr2=0x7fd5c419f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c37fe700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 0x7fd5c419f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.377+0000 7fd5c37fe700 1 -- 172.21.15.87:0/3622204231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c4101250 con 0x7fd5c4103460 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5c3fff700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5c37fe700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 0x7fd5c4198880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800ea20 tx=0x7fd5b800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b800cbd0 con 0x7fd5c4103460 2024-03-21T08:56:43.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd5b800ee60 con 0x7fd5c4103460 2024-03-21T08:56:43.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c4101540 con 0x7fd5c4103460 2024-03-21T08:56:43.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8018680 con 0x7fd5c4103460 2024-03-21T08:56:43.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.378+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c4101a80 con 0x7fd5c4103460 2024-03-21T08:56:43.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.379+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c4068e50 con 0x7fd5c4103460 2024-03-21T08:56:43.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.380+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd5b8018820 con 0x7fd5c4103460 2024-03-21T08:56:43.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.380+0000 7fd5c17fa700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd5ac0a2fb0 0x7fd5ac0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:43.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.380+0000 7fd5c3fff700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd5ac0a2fb0 0x7fd5ac0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:43.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.381+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd5b80cf4c0 con 0x7fd5c4103460 2024-03-21T08:56:43.378 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.381+0000 7fd5c3fff700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd5ac0a2fb0 0x7fd5ac0a5470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4005950 tx=0x7fd5b4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:43.381 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.385+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd5b809c320 con 0x7fd5c4103460 2024-03-21T08:56:43.590 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.594+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5c4109110 con 0x7fd5ac0a2fb0 2024-03-21T08:56:43.596 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.600+0000 7fd5c17fa700 1 -- 172.21.15.87:0/3622204231 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+980 (secure 0 0 0) 0x7fd5c4109110 con 0x7fd5ac0a2fb0 2024-03-21T08:56:43.596 INFO:teuthology.orchestra.run.smithi087.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-03-21T08:56:43.596 INFO:teuthology.orchestra.run.smithi087.stdout:mds.all 12/12 37s ago 2m smithi087=a;smithi087=d;smithi087=g;smithi087=j;smithi100=b;smithi100=e;smithi100=h;smithi100=k;smithi105=c;smithi105=f;smithi105=i;smithi105=l;count:12 2024-03-21T08:56:43.596 INFO:teuthology.orchestra.run.smithi087.stdout:mgr 3/3 37s ago 7m smithi087=x;smithi100=y;smithi105=z;count:3 2024-03-21T08:56:43.596 INFO:teuthology.orchestra.run.smithi087.stdout:mon 3/3 37s ago 8m smithi087:172.21.15.87=a;smithi100:172.21.15.100=b;smithi105:172.21.15.105=c;count:3 2024-03-21T08:56:43.597 INFO:teuthology.orchestra.run.smithi087.stdout:osd 12 37s ago - 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.602+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd5ac0a2fb0 msgr2=0x7fd5ac0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.602+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd5ac0a2fb0 0x7fd5ac0a5470 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd5b4005950 tx=0x7fd5b4009260 comp rx=0 tx=0).stop 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 msgr2=0x7fd5c4198880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 0x7fd5c4198880 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd5b800ea20 tx=0x7fd5b800ed30 comp rx=0 tx=0).stop 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 shutdown_connections 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd5ac0a2fb0 0x7fd5ac0a5470 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd5c4100480 0x7fd5c4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd5c4103e00 0x7fd5c419f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 --2- 172.21.15.87:0/3622204231 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd5c4103460 0x7fd5c4198880 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:43.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 >> 172.21.15.87:0/3622204231 conn(0x7fd5c40fbe00 msgr2=0x7fd5c4106760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:43.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 shutdown_connections 2024-03-21T08:56:43.600 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:43.603+0000 7fd5ca330700 1 -- 172.21.15.87:0/3622204231 wait complete. 2024-03-21T08:56:44.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:43 smithi105 ceph-mon[35993]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:56:44.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:43 smithi100 ceph-mon[36306]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:56:44.375 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:43 smithi087 ceph-mon[32385]: pgmap v340: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:56:44.404 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph orch host ls' 2024-03-21T08:56:44.646 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:45.096 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:44 smithi087 ceph-mon[32385]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:44 smithi105 ceph-mon[35993]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:45.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:44 smithi100 ceph-mon[36306]: from='client.24991 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:46.089 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:45 smithi087 ceph-mon[32385]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T08:56:46.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:45 smithi105 ceph-mon[35993]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T08:56:46.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:45 smithi100 ceph-mon[36306]: pgmap v341: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-21T08:56:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/1503066297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41022c0 msgr2=0x7fd1b4102720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/1503066297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41022c0 0x7fd1b4102720 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fd19c0099c0 tx=0x7fd19c009cd0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/1503066297 shutdown_connections 2024-03-21T08:56:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/1503066297 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102c60 0x7fd1b4109510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/1503066297 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41010c0 0x7fd1b41014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/1503066297 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41022c0 0x7fd1b4102720 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/1503066297 >> 172.21.15.87:0/1503066297 conn(0x7fd1b40fc660 msgr2=0x7fd1b40feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/1503066297 shutdown_connections 2024-03-21T08:56:46.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.327+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/1503066297 wait complete. 2024-03-21T08:56:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 Processor -- start 2024-03-21T08:56:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 -- start start 2024-03-21T08:56:46.325 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 0x7fd1b406aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41022c0 0x7fd1b419d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102c60 0x7fd1b419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410cc80 con 0x7fd1b41010c0 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410cb00 con 0x7fd1b4102c60 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1ba0dd700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b410ce00 con 0x7fd1b41022c0 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 0x7fd1b406aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 0x7fd1b406aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45734/0 (socket says 172.21.15.87:45734) 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 -- 172.21.15.87:0/3870434089 learned_addr learned my addr 172.21.15.87:0/3870434089 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1aaffd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41022c0 0x7fd1b419d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:46.326 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b3fff700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102c60 0x7fd1b419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 -- 172.21.15.87:0/3870434089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41022c0 msgr2=0x7fd1b419d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41022c0 0x7fd1b419d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 -- 172.21.15.87:0/3870434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102c60 msgr2=0x7fd1b419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102c60 0x7fd1b419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.328+0000 7fd1b37fe700 1 -- 172.21.15.87:0/3870434089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b41a0080 con 0x7fd1b41010c0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.329+0000 7fd1b37fe700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 0x7fd1b406aa00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400d980 tx=0x7fd1a400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.329+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4016070 con 0x7fd1b41010c0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.329+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fd1a400fd70 con 0x7fd1b41010c0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.329+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1b41a0370 con 0x7fd1b41010c0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.329+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a4019360 con 0x7fd1b41010c0 2024-03-21T08:56:46.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.329+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1b41a08b0 con 0x7fd1b41010c0 2024-03-21T08:56:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.330+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1b4068e50 con 0x7fd1b41010c0 2024-03-21T08:56:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.331+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fd1a4019070 con 0x7fd1b41010c0 2024-03-21T08:56:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.332+0000 7fd1b17fa700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd1a00a2fb0 0x7fd1a00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.332+0000 7fd1aaffd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd1a00a2fb0 0x7fd1a00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.332+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fd1a40ceae0 con 0x7fd1b41010c0 2024-03-21T08:56:46.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.332+0000 7fd1aaffd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd1a00a2fb0 0x7fd1a00a5470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd19c000c00 tx=0x7fd19c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:46.332 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.336+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fd1a409b940 con 0x7fd1b41010c0 2024-03-21T08:56:46.536 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.540+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd1b40625d0 con 0x7fd1a00a2fb0 2024-03-21T08:56:46.537 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.541+0000 7fd1b17fa700 1 -- 172.21.15.87:0/3870434089 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fd1b40625d0 con 0x7fd1a00a2fb0 2024-03-21T08:56:46.537 INFO:teuthology.orchestra.run.smithi087.stdout:HOST ADDR LABELS STATUS 2024-03-21T08:56:46.537 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087 172.21.15.87 2024-03-21T08:56:46.537 INFO:teuthology.orchestra.run.smithi087.stdout:smithi100 172.21.15.100 2024-03-21T08:56:46.537 INFO:teuthology.orchestra.run.smithi087.stdout:smithi105 172.21.15.105 2024-03-21T08:56:46.537 INFO:teuthology.orchestra.run.smithi087.stdout:3 hosts in cluster 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.543+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd1a00a2fb0 msgr2=0x7fd1a00a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.543+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd1a00a2fb0 0x7fd1a00a5470 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd19c000c00 tx=0x7fd19c005fb0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.543+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 msgr2=0x7fd1b406aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.543+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 0x7fd1b406aa00 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fd1a400d980 tx=0x7fd1a400dc90 comp rx=0 tx=0).stop 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 shutdown_connections 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fd1a00a2fb0 0x7fd1a00a5470 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fd1b4102c60 0x7fd1b419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fd1b41010c0 0x7fd1b406aa00 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 --2- 172.21.15.87:0/3870434089 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fd1b41022c0 0x7fd1b419d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 >> 172.21.15.87:0/3870434089 conn(0x7fd1b40fc660 msgr2=0x7fd1b41055e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 shutdown_connections 2024-03-21T08:56:46.540 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:46.544+0000 7fd1ba0dd700 1 -- 172.21.15.87:0/3870434089 wait complete. 2024-03-21T08:56:47.292 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph orch device ls' 2024-03-21T08:56:47.536 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:48.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:47 smithi105 ceph-mon[35993]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:48.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:47 smithi105 ceph-mon[35993]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:48.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:47 smithi100 ceph-mon[36306]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:48.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:47 smithi100 ceph-mon[36306]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:48.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:47 smithi087 ceph-mon[32385]: from='client.15174 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:48.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:47 smithi087 ceph-mon[32385]: pgmap v342: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3387729629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc0100be0 msgr2=0x7f6cc010a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3387729629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc0100be0 0x7f6cc010a7b0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6cb800b0a0 tx=0x7f6cb800b3b0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3387729629 shutdown_connections 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3387729629 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc01037c0 0x7f6cc0103be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3387729629 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc0100be0 0x7f6cc010a7b0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3387729629 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100240 0x7f6cc01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3387729629 >> 172.21.15.87:0/3387729629 conn(0x7f6cc00fbde0 msgr2=0x7f6cc00fe240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3387729629 shutdown_connections 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.211+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3387729629 wait complete. 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 Processor -- start 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 -- start start 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc0100240 0x7f6cc0102ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc01037c0 0x7f6cc0101a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc010dde0 con 0x7f6cc01037c0 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc010dc60 con 0x7f6cc0100240 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc81e1700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc010df60 con 0x7f6cc0100be0 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc577c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc677e700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc01037c0 0x7f6cc0101a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc577c700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:33926/0 (socket says 172.21.15.87:33926) 2024-03-21T08:56:49.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc577c700 1 -- 172.21.15.87:0/3309203855 learned_addr learned my addr 172.21.15.87:0/3309203855 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc5f7d700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc0100240 0x7f6cc0102ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc677e700 1 -- 172.21.15.87:0/3309203855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100be0 msgr2=0x7f6cc0101510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.212+0000 7f6cc677e700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cc677e700 1 -- 172.21.15.87:0/3309203855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc0100240 msgr2=0x7f6cc0102ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cc677e700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc0100240 0x7f6cc0102ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cc677e700 1 -- 172.21.15.87:0/3309203855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc0102300 con 0x7f6cc01037c0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cc677e700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc01037c0 0x7f6cc0101a50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009bc0 tx=0x7f6cb8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb8021070 con 0x7f6cc01037c0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc01a2490 con 0x7f6cc01037c0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6cb8006040 con 0x7f6cc01037c0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb80052a0 con 0x7f6cc01037c0 2024-03-21T08:56:49.211 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.213+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc01a29d0 con 0x7f6cc01037c0 2024-03-21T08:56:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.214+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc0068e50 con 0x7f6cc01037c0 2024-03-21T08:56:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.215+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6cb8024070 con 0x7f6cc01037c0 2024-03-21T08:56:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.215+0000 7f6cb6ffd700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6cac0a2fb0 0x7f6cac0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:49.212 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.216+0000 7f6cc5f7d700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6cac0a2fb0 0x7f6cac0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:49.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.216+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f6cb800f020 con 0x7f6cc01037c0 2024-03-21T08:56:49.213 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.216+0000 7f6cc5f7d700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6cac0a2fb0 0x7f6cac0a5470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6cc006a630 tx=0x7f6cb0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:49.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.220+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f6cb802b020 con 0x7f6cc01037c0 2024-03-21T08:56:49.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.423+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cc00625d0 con 0x7f6cac0a2fb0 2024-03-21T08:56:49.420 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.425+0000 7f6cb6ffd700 1 -- 172.21.15.87:0/3309203855 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+532 (secure 0 0 0) 0x7f6cc00625d0 con 0x7f6cac0a2fb0 2024-03-21T08:56:49.421 INFO:teuthology.orchestra.run.smithi087.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-03-21T08:56:49.421 INFO:teuthology.orchestra.run.smithi087.stdout:smithi087 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NQ400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-03-21T08:56:49.421 INFO:teuthology.orchestra.run.smithi087.stdout:smithi100 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400WR400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-21T08:56:49.421 INFO:teuthology.orchestra.run.smithi087.stdout:smithi105 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400TR400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-03-21T08:56:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6cac0a2fb0 msgr2=0x7f6cac0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6cac0a2fb0 0x7f6cac0a5470 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6cc006a630 tx=0x7f6cb0009260 comp rx=0 tx=0).stop 2024-03-21T08:56:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc01037c0 msgr2=0x7f6cc0101a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc01037c0 0x7f6cc0101a50 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8009bc0 tx=0x7f6cb8004d00 comp rx=0 tx=0).stop 2024-03-21T08:56:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 shutdown_connections 2024-03-21T08:56:49.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f6cac0a2fb0 0x7f6cac0a5470 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f6cc0100240 0x7f6cc0102ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f6cc01037c0 0x7f6cc0101a50 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 --2- 172.21.15.87:0/3309203855 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f6cc0100be0 0x7f6cc0101510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:49.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 >> 172.21.15.87:0/3309203855 conn(0x7f6cc00fbde0 msgr2=0x7f6cc0106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:49.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 shutdown_connections 2024-03-21T08:56:49.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:49.427+0000 7f6cc81e1700 1 -- 172.21.15.87:0/3309203855 wait complete. 2024-03-21T08:56:50.104 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:49 smithi087 ceph-mon[32385]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:50.130 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-03-21T08:56:50.140 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-03-21T08:56:50.140 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph fs dump' 2024-03-21T08:56:50.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:49 smithi105 ceph-mon[35993]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:50.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:49 smithi100 ceph-mon[36306]: pgmap v343: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:50.380 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:51.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:50 smithi087 ceph-mon[32385]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:51.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:50 smithi105 ceph-mon[35993]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:51.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:50 smithi100 ceph-mon[36306]: from='client.15180 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:56:52.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 -- 172.21.15.87:0/1321482187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 msgr2=0x7f84b410a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:52.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 --2- 172.21.15.87:0/1321482187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b410a630 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b0d0 tx=0x7f84ac00b3e0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 -- 172.21.15.87:0/1321482187 shutdown_connections 2024-03-21T08:56:52.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 --2- 172.21.15.87:0/1321482187 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b410a630 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.169 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 --2- 172.21.15.87:0/1321482187 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b41021e0 0x7f84b4102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 --2- 172.21.15.87:0/1321482187 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 0x7f84b4103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 -- 172.21.15.87:0/1321482187 >> 172.21.15.87:0/1321482187 conn(0x7f84b40fd7a0 msgr2=0x7f84b40ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:52.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 -- 172.21.15.87:0/1321482187 shutdown_connections 2024-03-21T08:56:52.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 -- 172.21.15.87:0/1321482187 wait complete. 2024-03-21T08:56:52.170 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.173+0000 7f84b9839700 1 Processor -- start 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 -- start start 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b41021e0 0x7f84b4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 0x7f84b41988a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b419f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b410dde0 con 0x7f84b4103d80 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b410dc60 con 0x7f84b41033e0 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b9839700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84b410df60 con 0x7f84b41021e0 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b419f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b2ffd700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b41021e0 0x7f84b4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b419f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51362/0 (socket says 172.21.15.87:51362) 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 -- 172.21.15.87:0/2420383569 learned_addr learned my addr 172.21.15.87:0/2420383569 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:52.172 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b27fc700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 0x7f84b41988a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 -- 172.21.15.87:0/2420383569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b41021e0 msgr2=0x7f84b4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b41021e0 0x7f84b4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 -- 172.21.15.87:0/2420383569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 msgr2=0x7f84b41988a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 0x7f84b41988a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 -- 172.21.15.87:0/2420383569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b419ffc0 con 0x7f84b4103d80 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b27fc700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 0x7f84b41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.174+0000 7f84b37fe700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b419f920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b770 tx=0x7f84ac007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.175+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac003c00 con 0x7f84b4103d80 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.175+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f84ac022070 con 0x7f84b4103d80 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.175+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac01b750 con 0x7f84b4103d80 2024-03-21T08:56:52.173 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.175+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b41a0250 con 0x7f84b4103d80 2024-03-21T08:56:52.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.175+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84b41a06a0 con 0x7f84b4103d80 2024-03-21T08:56:52.174 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.176+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84b4068e50 con 0x7f84b4103d80 2024-03-21T08:56:52.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.177+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f84ac01d470 con 0x7f84b4103d80 2024-03-21T08:56:52.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.177+0000 7f84b8837700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f849c0a3030 0x7f849c0a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:52.175 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.177+0000 7f84b2ffd700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f849c0a3030 0x7f849c0a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:52.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.177+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f84ac0d0330 con 0x7f84b4103d80 2024-03-21T08:56:52.176 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.178+0000 7f84b2ffd700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f849c0a3030 0x7f849c0a54f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84b4103240 tx=0x7f84a8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:52.177 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.182+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f84ac09d220 con 0x7f84b4103d80 2024-03-21T08:56:52.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:51 smithi105 ceph-mon[35993]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:52.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:51 smithi100 ceph-mon[36306]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:52.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:51 smithi087 ceph-mon[32385]: pgmap v344: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:52.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.422+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f84b4062da0 con 0x7f84b4103d80 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.423+0000 7f84b8837700 1 -- 172.21.15.87:0/2420383569 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+3037 (secure 0 0 0) 0x7f84ac09cb20 con 0x7f84b4103d80 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:e25 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:legacy client fscid: 1 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:Filesystem 'cephfs' (1) 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:fs_name cephfs 2024-03-21T08:56:52.419 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 25 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-03-21T08:55:06.567291+0000 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:modified 2024-03-21T08:55:12.955598+0000 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:tableserver 0 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:root 0 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:session_timeout 300 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:session_autoclose 300 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:max_file_size 1099511627776 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:max_xattr_size 65536 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:required_client_features {} 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:last_failure 0 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:last_failure_osd_epoch 0 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-21T08:56:52.420 INFO:teuthology.orchestra.run.smithi087.stdout:max_mds 5 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:in 0,1,2,3,4 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:up {0=24487,1=24482,2=24451,3=24425,4=24403} 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:failed 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:damaged 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:stopped 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:data_pools [3] 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:metadata_pool 2 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:inline_data disabled 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:balancer 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:bal_rank_mask -1 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:standby_count_wanted 0 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:qdb_cluster leader: 24487 members: 24451,24482,24425,24487,24403 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.i{0:24487} state up:active seq 5 addr [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.c{0:24469} state up:standby-replay seq 1 addr [v2:172.21.15.105:6836/2656551527,v1:172.21.15.105:6837/2656551527] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.421 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.b{1:24482} state up:active seq 4 addr [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.h{1:24455} state up:standby-replay seq 1 addr [v2:172.21.15.100:6836/2948361419,v1:172.21.15.100:6837/2948361419] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.f{2:24451} state up:active seq 8 addr [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.k{2:24395} state up:standby-replay seq 1 addr [v2:172.21.15.100:6832/1637520984,v1:172.21.15.100:6833/1637520984] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.e{3:24425} state up:active seq 9 addr [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.a{3:14646} state up:standby-replay seq 1 addr [v2:172.21.15.87:6840/2007269086,v1:172.21.15.87:6841/2007269086] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.l{4:24403} state up:active seq 12 addr [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.g{4:14634} state up:standby-replay seq 1 addr [v2:172.21.15.87:6838/3694307942,v1:172.21.15.87:6839/3694307942] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:Standby daemons: 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.j{-1:14553} state up:standby seq 1 addr [v2:172.21.15.87:6834/2095083606,v1:172.21.15.87:6835/2095083606] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.422 INFO:teuthology.orchestra.run.smithi087.stdout:[mds.d{-1:14565} state up:standby seq 1 addr [v2:172.21.15.87:6836/4247247896,v1:172.21.15.87:6837/4247247896] compat {c=[1],r=[1],i=[fff]}] 2024-03-21T08:56:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f849c0a3030 msgr2=0x7f849c0a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:52.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f849c0a3030 0x7f849c0a54f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f84b4103240 tx=0x7f84a8009260 comp rx=0 tx=0).stop 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 msgr2=0x7f84b419f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b419f920 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f84ac00b770 tx=0x7f84ac007a00 comp rx=0 tx=0).stop 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 shutdown_connections 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f849c0a3030 0x7f849c0a54f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f84b4103d80 0x7f84b419f920 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f84b41021e0 0x7f84b4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 --2- 172.21.15.87:0/2420383569 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f84b41033e0 0x7f84b41988a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 >> 172.21.15.87:0/2420383569 conn(0x7f84b40fd7a0 msgr2=0x7f84b4106680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 shutdown_connections 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:52.426+0000 7f84b9839700 1 -- 172.21.15.87:0/2420383569 wait complete. 2024-03-21T08:56:52.424 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:56:53.092 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- bash -c 'ceph osd dump' 2024-03-21T08:56:53.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:52 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2420383569' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T08:56:53.341 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:56:53.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:52 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2420383569' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T08:56:53.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:52 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2420383569' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-03-21T08:56:54.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:53 smithi087 ceph-mon[32385]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:54.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:53 smithi100 ceph-mon[36306]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:54.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:53 smithi105 ceph-mon[35993]: pgmap v345: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 -- 172.21.15.87:0/145685728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d440fff30 msgr2=0x7f4d44100350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 --2- 172.21.15.87:0/145685728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d440fff30 0x7f4d44100350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f4d34009990 tx=0x7f4d34009ca0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 -- 172.21.15.87:0/145685728 shutdown_connections 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 --2- 172.21.15.87:0/145685728 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44107e70 0x7f4d4410a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 --2- 172.21.15.87:0/145685728 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d440fff30 0x7f4d44100350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 --2- 172.21.15.87:0/145685728 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d44100890 0x7f4d441078a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 -- 172.21.15.87:0/145685728 >> 172.21.15.87:0/145685728 conn(0x7f4d440fb990 msgr2=0x7f4d440fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 -- 172.21.15.87:0/145685728 shutdown_connections 2024-03-21T08:56:55.223 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.224+0000 7f4d49006700 1 -- 172.21.15.87:0/145685728 wait complete. 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.225+0000 7f4d49006700 1 Processor -- start 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.225+0000 7f4d49006700 1 -- start start 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d49006700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d440fff30 0x7f4d44198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d49006700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 0x7f4d44198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d49006700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44107e70 0x7f4d4419f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d49006700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4410ddd0 con 0x7f4d44107e70 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d49006700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4410dc50 con 0x7f4d44100890 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d49006700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d4410df50 con 0x7f4d440fff30 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d42d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d440fff30 0x7f4d44198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 0x7f4d44198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d42d9d700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d440fff30 0x7f4d44198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:33968/0 (socket says 172.21.15.87:33968) 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d42d9d700 1 -- 172.21.15.87:0/4007598473 learned_addr learned my addr 172.21.15.87:0/4007598473 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:55.224 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 0x7f4d44198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:51452/0 (socket says 172.21.15.87:51452) 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4359e700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44107e70 0x7f4d4419f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 -- 172.21.15.87:0/4007598473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d440fff30 msgr2=0x7f4d44198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d440fff30 0x7f4d44198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 -- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44107e70 msgr2=0x7f4d4419f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44107e70 0x7f4d4419f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 -- 172.21.15.87:0/4007598473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d441a0070 con 0x7f4d44100890 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d4259c700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 0x7f4d44198950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d44100cb0 tx=0x7f4d3800eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.226+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38010900 con 0x7f4d44100890 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.227+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f4d3800c430 con 0x7f4d44100890 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.227+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d441a0360 con 0x7f4d44100890 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.227+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d38018660 con 0x7f4d44100890 2024-03-21T08:56:55.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.227+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d441a08a0 con 0x7f4d44100890 2024-03-21T08:56:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.227+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d44068e50 con 0x7f4d44100890 2024-03-21T08:56:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.229+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f4d3800c950 con 0x7f4d44100890 2024-03-21T08:56:55.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.229+0000 7f4d2bfff700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f4d2c0a2fb0 0x7f4d2c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:55.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.229+0000 7f4d42d9d700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f4d2c0a2fb0 0x7f4d2c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:55.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.229+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f4d380cf9a0 con 0x7f4d44100890 2024-03-21T08:56:55.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.230+0000 7f4d42d9d700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f4d2c0a2fb0 0x7f4d2c0a5470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400b2e0 tx=0x7f4d3400b880 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:55.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.233+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f4d3809c7a0 con 0x7f4d44100890 2024-03-21T08:56:55.423 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.427+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7f4d4402da30 con 0x7f4d44100890 2024-03-21T08:56:55.426 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.428+0000 7f4d2bfff700 1 -- 172.21.15.87:0/4007598473 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v88) v1 ==== 56+0+5260 (secure 0 0 0) 0x7f4d3809c0a0 con 0x7f4d44100890 2024-03-21T08:56:55.426 INFO:teuthology.orchestra.run.smithi087.stdout:epoch 88 2024-03-21T08:56:55.426 INFO:teuthology.orchestra.run.smithi087.stdout:fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T08:56:55.426 INFO:teuthology.orchestra.run.smithi087.stdout:created 2024-03-21T08:47:04.720561+0000 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:modified 2024-03-21T08:56:00.061159+0000 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:crush_version 26 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:full_ratio 0.95 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:backfillfull_ratio 0.9 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:nearfull_ratio 0.85 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:require_min_compat_client luminous 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:min_compat_client luminous 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:require_osd_release squid 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:stretch_mode_enabled false 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-03-21T08:56:55.427 INFO:teuthology.orchestra.run.smithi087.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 86 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:max_osd 12 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.0 up in weight 1 up_from 8 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6802/3363556070,v1:172.21.15.87:6803/3363556070] [v2:172.21.15.87:6804/3363556070,v1:172.21.15.87:6805/3363556070] exists,up 399009d1-377c-4c98-80d9-f0b22744ad40 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6800/4136971528,v1:172.21.15.100:6801/4136971528] [v2:172.21.15.100:6802/4136971528,v1:172.21.15.100:6803/4136971528] exists,up f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.2 up in weight 1 up_from 18 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6800/3040563214,v1:172.21.15.105:6801/3040563214] [v2:172.21.15.105:6802/3040563214,v1:172.21.15.105:6803/3040563214] exists,up 90e5dc9a-effc-4556-a9a4-b84e24ae3338 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.3 up in weight 1 up_from 25 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6810/269698093,v1:172.21.15.87:6811/269698093] [v2:172.21.15.87:6812/269698093,v1:172.21.15.87:6813/269698093] exists,up bd94ec48-89ca-41e0-a0af-f91f7d8d3d14 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.4 up in weight 1 up_from 33 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6808/3040469961,v1:172.21.15.100:6809/3040469961] [v2:172.21.15.100:6810/3040469961,v1:172.21.15.100:6811/3040469961] exists,up 8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.5 up in weight 1 up_from 39 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6808/1039939293,v1:172.21.15.105:6809/1039939293] [v2:172.21.15.105:6810/1039939293,v1:172.21.15.105:6811/1039939293] exists,up 4f908d67-699b-4d6f-8543-3f5dcaed7fcd 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.6 up in weight 1 up_from 47 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6818/944234078,v1:172.21.15.87:6819/944234078] [v2:172.21.15.87:6820/944234078,v1:172.21.15.87:6821/944234078] exists,up fa673f89-d652-4f14-9f54-c034e053a748 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.7 up in weight 1 up_from 52 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6816/1198353584,v1:172.21.15.100:6817/1198353584] [v2:172.21.15.100:6818/1198353584,v1:172.21.15.100:6819/1198353584] exists,up 75a7a975-6491-4d37-b314-c75617e3b0cc 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.8 up in weight 1 up_from 60 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6816/2122777536,v1:172.21.15.105:6817/2122777536] [v2:172.21.15.105:6818/2122777536,v1:172.21.15.105:6819/2122777536] exists,up bd46697a-3872-410d-b1ab-63fa68278d79 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.9 up in weight 1 up_from 68 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.87:6826/3198171550,v1:172.21.15.87:6827/3198171550] [v2:172.21.15.87:6828/3198171550,v1:172.21.15.87:6829/3198171550] exists,up 2aee04df-d0a2-48c3-89b5-eb79810633c0 2024-03-21T08:56:55.428 INFO:teuthology.orchestra.run.smithi087.stdout:osd.10 up in weight 1 up_from 73 up_thru 84 down_at 0 last_clean_interval [0,0) [v2:172.21.15.100:6824/1731228246,v1:172.21.15.100:6825/1731228246] [v2:172.21.15.100:6826/1731228246,v1:172.21.15.100:6827/1731228246] exists,up cbfb1970-a260-4470-b56b-eff1363dd1b5 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:osd.11 up in weight 1 up_from 81 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.105:6824/3636544380,v1:172.21.15.105:6825/3636544380] [v2:172.21.15.105:6826/3636544380,v1:172.21.15.105:6827/3636544380] exists,up d12fe1b9-2770-49d6-a7ca-d6595f7041f2 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 2.8 [2,11] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 2.f [8,11] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.13 [4,9] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.14 [4,6] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.18 [3,9] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.2c [4,9] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:pg_upmap_items 3.37 [5,8] 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6801/2393104803 expires 2024-03-22T08:47:58.771455+0000 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6800/2393104803 expires 2024-03-22T08:47:58.771455+0000 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/845087826 expires 2024-03-22T08:47:58.771455+0000 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/4284800912 expires 2024-03-22T08:47:31.789576+0000 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/3681410836 expires 2024-03-22T08:47:31.789576+0000 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/215090592 expires 2024-03-22T08:47:31.789576+0000 2024-03-21T08:56:55.429 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6800/277415650 expires 2024-03-22T08:47:31.789576+0000 2024-03-21T08:56:55.430 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/3180053718 expires 2024-03-22T08:47:58.771455+0000 2024-03-21T08:56:55.430 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:6801/277415650 expires 2024-03-22T08:47:31.789576+0000 2024-03-21T08:56:55.430 INFO:teuthology.orchestra.run.smithi087.stdout:blocklist 172.21.15.87:0/3000671039 expires 2024-03-22T08:47:58.771455+0000 2024-03-21T08:56:55.430 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f4d2c0a2fb0 msgr2=0x7f4d2c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f4d2c0a2fb0 0x7f4d2c0a5470 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4d3400b2e0 tx=0x7f4d3400b880 comp rx=0 tx=0).stop 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 msgr2=0x7f4d44198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 0x7f4d44198950 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d44100cb0 tx=0x7f4d3800eca0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 shutdown_connections 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f4d2c0a2fb0 0x7f4d2c0a5470 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f4d44107e70 0x7f4d4419f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f4d440fff30 0x7f4d44198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 --2- 172.21.15.87:0/4007598473 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f4d44100890 0x7f4d44198950 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.430+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 >> 172.21.15.87:0/4007598473 conn(0x7f4d440fb990 msgr2=0x7f4d44103880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.431+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 shutdown_connections 2024-03-21T08:56:55.431 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:55.431+0000 7f4d49006700 1 -- 172.21.15.87:0/4007598473 wait complete. 2024-03-21T08:56:56.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:55 smithi087 ceph-mon[32385]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:56.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:55 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/4007598473' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T08:56:56.247 INFO:teuthology.run_tasks:Running task fs.ready... 2024-03-21T08:56:56.258 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:56:56.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:55 smithi100 ceph-mon[36306]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:56.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:55 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/4007598473' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T08:56:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 -- 172.21.15.87:0/3496885963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781003f0 msgr2=0x7f0378100810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 --2- 172.21.15.87:0/3496885963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781003f0 0x7f0378100810 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f036c009900 tx=0x7f036c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 -- 172.21.15.87:0/3496885963 shutdown_connections 2024-03-21T08:56:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 --2- 172.21.15.87:0/3496885963 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0378101fa0 0x7f0378108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 --2- 172.21.15.87:0/3496885963 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f03781015e0 0x7f0378101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.368 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 --2- 172.21.15.87:0/3496885963 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781003f0 0x7f0378100810 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 -- 172.21.15.87:0/3496885963 >> 172.21.15.87:0/3496885963 conn(0x7f03780fb980 msgr2=0x7f03780fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 -- 172.21.15.87:0/3496885963 shutdown_connections 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 -- 172.21.15.87:0/3496885963 wait complete. 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.370+0000 7f037db9d640 1 Processor -- start 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 -- start start 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03781003f0 0x7f0378196730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 0x7f0378196c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0378101fa0 0x7f037819dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037810c080 con 0x7f03781015e0 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037810bf00 con 0x7f0378101fa0 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037810c200 con 0x7f03781003f0 2024-03-21T08:56:56.369 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 0x7f0378196c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037cb9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03781003f0 0x7f0378196730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 0x7f0378196c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51388/0 (socket says 172.21.15.87:51388) 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 -- 172.21.15.87:0/3189202357 learned_addr learned my addr 172.21.15.87:0/3189202357 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037d39c640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0378101fa0 0x7f037819dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 -- 172.21.15.87:0/3189202357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03781003f0 msgr2=0x7f0378196730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03781003f0 0x7f0378196730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 -- 172.21.15.87:0/3189202357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0378101fa0 msgr2=0x7f037819dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0378101fa0 0x7f037819dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 -- 172.21.15.87:0/3189202357 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f037819e350 con 0x7f03781015e0 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037cb9b640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03781003f0 0x7f0378196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0377fff640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 0x7f0378196c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9f0 tx=0x7f036400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400cd80 con 0x7f03781015e0 2024-03-21T08:56:56.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f037819e640 con 0x7f03781015e0 2024-03-21T08:56:56.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0364010910 con 0x7f03781015e0 2024-03-21T08:56:56.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.371+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640188f0 con 0x7f03781015e0 2024-03-21T08:56:56.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.372+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f037819ebe0 con 0x7f03781015e0 2024-03-21T08:56:56.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.372+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0378107710 con 0x7f03781015e0 2024-03-21T08:56:56.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.374+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0364010ac0 con 0x7f03781015e0 2024-03-21T08:56:56.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.374+0000 7f0375ffb640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f03680a2f20 0x7f03680a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.374+0000 7f037cb9b640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f03680a2f20 0x7f03680a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.374+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f03640cfb10 con 0x7f03781015e0 2024-03-21T08:56:56.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.375+0000 7f037cb9b640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f03680a2f20 0x7f03680a53e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f036c000c00 tx=0x7f036c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:56.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.377+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f036409c320 con 0x7f03781015e0 2024-03-21T08:56:56.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:55 smithi105 ceph-mon[35993]: pgmap v346: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:56:56.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:55 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/4007598473' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-21T08:56:56.550 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.554+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f037802d270 con 0x7f03781015e0 2024-03-21T08:56:56.551 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.555+0000 7f0375ffb640 1 -- 172.21.15.87:0/3189202357 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7f03640a03c0 con 0x7f03781015e0 2024-03-21T08:56:56.552 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:56.552 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f03680a2f20 msgr2=0x7f03680a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f03680a2f20 0x7f03680a53e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f036c000c00 tx=0x7f036c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 msgr2=0x7f0378196c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 0x7f0378196c70 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f036400e9f0 tx=0x7f036400eeb0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 shutdown_connections 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f03680a2f20 0x7f03680a53e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0378101fa0 0x7f037819dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.554 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f03781015e0 0x7f0378196c70 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 --2- 172.21.15.87:0/3189202357 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f03781003f0 0x7f0378196730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 >> 172.21.15.87:0/3189202357 conn(0x7f03780fb980 msgr2=0x7f03781048e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:56.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 shutdown_connections 2024-03-21T08:56:56.555 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.558+0000 7f037db9d640 1 -- 172.21.15.87:0/3189202357 wait complete. 2024-03-21T08:56:56.565 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:55:04.486296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:55:05.756309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:56:56.566 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 -- 172.21.15.87:0/1730603133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc102170 msgr2=0x7fc8dc108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 --2- 172.21.15.87:0/1730603133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc102170 0x7fc8dc108a00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009ca0 tx=0x7fc8d402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 -- 172.21.15.87:0/1730603133 shutdown_connections 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 --2- 172.21.15.87:0/1730603133 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc102170 0x7fc8dc108a00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 --2- 172.21.15.87:0/1730603133 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8dc1017b0 0x7fc8dc101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 --2- 172.21.15.87:0/1730603133 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc1009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.673 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 -- 172.21.15.87:0/1730603133 >> 172.21.15.87:0/1730603133 conn(0x7fc8dc0fbb50 msgr2=0x7fc8dc0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 -- 172.21.15.87:0/1730603133 shutdown_connections 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 -- 172.21.15.87:0/1730603133 wait complete. 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 Processor -- start 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.675+0000 7fc8e2753640 1 -- start start 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8dc1017b0 0x7fc8dc197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8dc102170 0x7fc8dc19ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc10c220 con 0x7fc8dc1005c0 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc10c0a0 con 0x7fc8dc102170 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc10c3a0 con 0x7fc8dc1017b0 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.674 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51402/0 (socket says 172.21.15.87:51402) 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 -- 172.21.15.87:0/2209012163 learned_addr learned my addr 172.21.15.87:0/2209012163 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8db7fe640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8dc1017b0 0x7fc8dc197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e0cf0640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8dc102170 0x7fc8dc19ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 -- 172.21.15.87:0/2209012163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8dc1017b0 msgr2=0x7fc8dc197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8dc1017b0 0x7fc8dc197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 -- 172.21.15.87:0/2209012163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8dc102170 msgr2=0x7fc8dc19ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8dc102170 0x7fc8dc19ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 -- 172.21.15.87:0/2209012163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8dc19f580 con 0x7fc8dc1005c0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8dbfff640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc197960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00e9c0 tx=0x7fc8cc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc00cd80 con 0x7fc8dc1005c0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fc8cc0035f0 con 0x7fc8dc1005c0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc19f870 con 0x7fc8dc1005c0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8cc010690 con 0x7fc8dc1005c0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.676+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc19fdc0 con 0x7fc8dc1005c0 2024-03-21T08:56:56.675 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.677+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc1a0020 con 0x7fc8dc1005c0 2024-03-21T08:56:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.678+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fc8cc0037a0 con 0x7fc8dc1005c0 2024-03-21T08:56:56.676 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.679+0000 7fc8d97fa640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8c40a2f20 0x7fc8c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.679+0000 7fc8db7fe640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8c40a2f20 0x7fc8c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.679+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fc8cc0cf850 con 0x7fc8dc1005c0 2024-03-21T08:56:56.677 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.679+0000 7fc8db7fe640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8c40a2f20 0x7fc8c40a53e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc198e70 tx=0x7fc8d0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:56.678 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.682+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fc8cc09c740 con 0x7fc8dc1005c0 2024-03-21T08:56:56.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.857+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc8dc1a03b0 con 0x7fc8dc1005c0 2024-03-21T08:56:56.853 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.857+0000 7fc8d97fa640 1 -- 172.21.15.87:0/2209012163 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc8cc09c040 con 0x7fc8dc1005c0 2024-03-21T08:56:56.853 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:56:56.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8c40a2f20 msgr2=0x7fc8c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8c40a2f20 0x7fc8c40a53e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc198e70 tx=0x7fc8d0005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:56.856 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 msgr2=0x7fc8dc197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc197960 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc8cc00e9c0 tx=0x7fc8cc00ee80 comp rx=0 tx=0).stop 2024-03-21T08:56:56.857 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 shutdown_connections 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fc8c40a2f20 0x7fc8c40a53e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fc8dc102170 0x7fc8dc19ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fc8dc1017b0 0x7fc8dc197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 --2- 172.21.15.87:0/2209012163 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fc8dc1005c0 0x7fc8dc197960 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 >> 172.21.15.87:0/2209012163 conn(0x7fc8dc0fbb50 msgr2=0x7fc8dc069cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 shutdown_connections 2024-03-21T08:56:56.858 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.859+0000 7fc8e2753640 1 -- 172.21.15.87:0/2209012163 wait complete. 2024-03-21T08:56:56.867 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:56:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 -- 172.21.15.87:0/1775977288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1228103260 msgr2=0x7f1228109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1775977288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1228103260 0x7f1228109af0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1220009ca0 tx=0x7f122002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 -- 172.21.15.87:0/1775977288 shutdown_connections 2024-03-21T08:56:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1775977288 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1228103260 0x7f1228109af0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1775977288 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f12281028a0 0x7f1228102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.972 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1775977288 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12281016b0 0x7f1228101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 -- 172.21.15.87:0/1775977288 >> 172.21.15.87:0/1775977288 conn(0x7f12280fcc40 msgr2=0x7f12280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 -- 172.21.15.87:0/1775977288 shutdown_connections 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 -- 172.21.15.87:0/1775977288 wait complete. 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.974+0000 7f122e3b9640 1 Processor -- start 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 -- start start 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 0x7f1228197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12281028a0 0x7f1228197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228103260 0x7f122819ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122810d330 con 0x7f12281016b0 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122810d1b0 con 0x7f1228103260 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122e3b9640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122810d4b0 con 0x7f12281028a0 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 0x7f1228197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 0x7f1228197950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51418/0 (socket says 172.21.15.87:51418) 2024-03-21T08:56:56.973 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 -- 172.21.15.87:0/1381874280 learned_addr learned my addr 172.21.15.87:0/1381874280 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f12277fe640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12281028a0 0x7f1228197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122c956640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228103260 0x7f122819ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 -- 172.21.15.87:0/1381874280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12281028a0 msgr2=0x7f1228197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12281028a0 0x7f1228197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 -- 172.21.15.87:0/1381874280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228103260 msgr2=0x7f122819ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228103260 0x7f122819ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 -- 172.21.15.87:0/1381874280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122819f570 con 0x7f12281016b0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f122c956640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228103260 0x7f122819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.975+0000 7f1227fff640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 0x7f1228197950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f121800e9c0 tx=0x7f121800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.976+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121800cd80 con 0x7f12281016b0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.976+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122819f860 con 0x7f12281016b0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.976+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f12180035f0 con 0x7f12281016b0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.976+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1218010690 con 0x7f12281016b0 2024-03-21T08:56:56.974 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.976+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122819fdb0 con 0x7f12281016b0 2024-03-21T08:56:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.977+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12281089d0 con 0x7f12281016b0 2024-03-21T08:56:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.978+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f12180037a0 con 0x7f12281016b0 2024-03-21T08:56:56.975 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.978+0000 7f12257fa640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12100a2f20 0x7f12100a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.978+0000 7f12277fe640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12100a2f20 0x7f12100a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.978+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f12180cf720 con 0x7f12281016b0 2024-03-21T08:56:56.976 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.979+0000 7f12277fe640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12100a2f20 0x7f12100a53e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f121c0095f0 tx=0x7f121c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:56.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:56.983+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f121809c580 con 0x7f12281016b0 2024-03-21T08:56:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.159+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f12280622a0 con 0x7f12281016b0 2024-03-21T08:56:57.155 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.160+0000 7f12257fa640 1 -- 172.21.15.87:0/1381874280 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f121809be80 con 0x7f12281016b0 2024-03-21T08:56:57.156 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-03-21T08:56:57.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:57 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3189202357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:57.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:57 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2209012163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12100a2f20 msgr2=0x7f12100a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12100a2f20 0x7f12100a53e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f121c0095f0 tx=0x7f121c005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 msgr2=0x7f1228197950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 0x7f1228197950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f121800e9c0 tx=0x7f121800ee80 comp rx=0 tx=0).stop 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 shutdown_connections 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f12100a2f20 0x7f12100a53e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1228103260 0x7f122819ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f12281028a0 0x7f1228197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 --2- 172.21.15.87:0/1381874280 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f12281016b0 0x7f1228197950 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 >> 172.21.15.87:0/1381874280 conn(0x7f12280fcc40 msgr2=0x7f1228105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 shutdown_connections 2024-03-21T08:56:57.159 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.162+0000 7f122e3b9640 1 -- 172.21.15.87:0/1381874280 wait complete. 2024-03-21T08:56:57.169 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:56:57.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:57 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3189202357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:57.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:57 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2209012163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:57.273 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 -- 172.21.15.87:0/903267018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 msgr2=0x7ff6c0106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/903267018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0106e60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4009900 tx=0x7ff6b402bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 -- 172.21.15.87:0/903267018 shutdown_connections 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/903267018 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c01073a0 0x7ff6c01097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/903267018 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0106e60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/903267018 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c0102140 0x7ff6c0104530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 -- 172.21.15.87:0/903267018 >> 172.21.15.87:0/903267018 conn(0x7ff6c00fbb50 msgr2=0x7ff6c00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 -- 172.21.15.87:0/903267018 shutdown_connections 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.277+0000 7ff6c68b2640 1 -- 172.21.15.87:0/903267018 wait complete. 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 Processor -- start 2024-03-21T08:56:57.274 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 -- start start 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0102140 0x7ff6c0197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c01073a0 0x7ff6c019efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c010d350 con 0x7ff6c0104a70 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c010d1d0 con 0x7ff6c0102140 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c68b2640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6c010d4d0 con 0x7ff6c01073a0 2024-03-21T08:56:57.276 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:51440/0 (socket says 172.21.15.87:51440) 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 -- 172.21.15.87:0/2880135694 learned_addr learned my addr 172.21.15.87:0/2880135694 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6c4e4f640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c01073a0 0x7ff6c019efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bffff640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0102140 0x7ff6c0197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 -- 172.21.15.87:0/2880135694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c01073a0 msgr2=0x7ff6c019efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c01073a0 0x7ff6c019efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 -- 172.21.15.87:0/2880135694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0102140 msgr2=0x7ff6c0197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0102140 0x7ff6c0197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.278+0000 7ff6bf7fe640 1 -- 172.21.15.87:0/2880135694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c019f640 con 0x7ff6c0104a70 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.279+0000 7ff6bf7fe640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0197f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4000c00 tx=0x7ff6b402c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.279+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4041070 con 0x7ff6c0104a70 2024-03-21T08:56:57.277 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.279+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c019f8d0 con 0x7ff6c0104a70 2024-03-21T08:56:57.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.279+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff6b4004450 con 0x7ff6c0104a70 2024-03-21T08:56:57.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.279+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6b4034440 con 0x7ff6c0104a70 2024-03-21T08:56:57.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.279+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6c019fe10 con 0x7ff6c0104a70 2024-03-21T08:56:57.278 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.280+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6c00fd640 con 0x7ff6c0104a70 2024-03-21T08:56:57.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.281+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff6b4044070 con 0x7ff6c0104a70 2024-03-21T08:56:57.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.281+0000 7ff6bd7fa640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff6a80a2f20 0x7ff6a80a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.282+0000 7ff6bffff640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff6a80a2f20 0x7ff6a80a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.282+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff6b40f0fb0 con 0x7ff6c0104a70 2024-03-21T08:56:57.279 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.282+0000 7ff6bffff640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff6a80a2f20 0x7ff6a80a53e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009bd0 tx=0x7ff6b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:57.281 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.285+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff6b40bd8f0 con 0x7ff6c0104a70 2024-03-21T08:56:57.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.460+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6c002d070 con 0x7ff6c0104a70 2024-03-21T08:56:57.457 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.461+0000 7ff6bd7fa640 1 -- 172.21.15.87:0/2880135694 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6b40c1990 con 0x7ff6c0104a70 2024-03-21T08:56:57.457 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff6a80a2f20 msgr2=0x7ff6a80a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff6a80a2f20 0x7ff6a80a53e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ff6b0009bd0 tx=0x7ff6b0005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 msgr2=0x7ff6c0197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0197f60 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6b4000c00 tx=0x7ff6b402c640 comp rx=0 tx=0).stop 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 shutdown_connections 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff6a80a2f20 0x7ff6a80a53e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff6c01073a0 0x7ff6c019efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff6c0104a70 0x7ff6c0197f60 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 --2- 172.21.15.87:0/2880135694 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff6c0102140 0x7ff6c0197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 >> 172.21.15.87:0/2880135694 conn(0x7ff6c00fbb50 msgr2=0x7ff6c0102c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:57.459 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 shutdown_connections 2024-03-21T08:56:57.460 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.463+0000 7ff6c68b2640 1 -- 172.21.15.87:0/2880135694 wait complete. 2024-03-21T08:56:57.469 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:56:57.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:57 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3189202357' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:57.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:57 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2209012163' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 -- 172.21.15.87:0/3309419287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 msgr2=0x7f0cb4101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/3309419287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb4101af0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0ca4009ca0 tx=0x7f0ca402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 -- 172.21.15.87:0/3309419287 shutdown_connections 2024-03-21T08:56:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/3309419287 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cb4103280 0x7f0cb4109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.573 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/3309419287 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb41028c0 0x7f0cb4102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/3309419287 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb4101af0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 -- 172.21.15.87:0/3309419287 >> 172.21.15.87:0/3309419287 conn(0x7f0cb40fcc80 msgr2=0x7f0cb40ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 -- 172.21.15.87:0/3309419287 shutdown_connections 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.575+0000 7f0cb97f3640 1 -- 172.21.15.87:0/3309419287 wait complete. 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 Processor -- start 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 -- start start 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb41979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb41028c0 0x7f0cb4197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cb4103280 0x7f0cb419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb410d320 con 0x7f0cb41016d0 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb410d1a0 con 0x7f0cb41028c0 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb97f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb410d4a0 con 0x7f0cb4103280 2024-03-21T08:56:57.574 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb41979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb41979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58842/0 (socket says 172.21.15.87:58842) 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 -- 172.21.15.87:0/1309258191 learned_addr learned my addr 172.21.15.87:0/1309258191 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb27fc640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb41028c0 0x7f0cb4197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb37fe640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cb4103280 0x7f0cb419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 -- 172.21.15.87:0/1309258191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cb4103280 msgr2=0x7f0cb419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cb4103280 0x7f0cb419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 -- 172.21.15.87:0/1309258191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb41028c0 msgr2=0x7f0cb4197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb41028c0 0x7f0cb4197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.576+0000 7f0cb2ffd640 1 -- 172.21.15.87:0/1309258191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb419f600 con 0x7f0cb41016d0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.577+0000 7f0cb2ffd640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb41979e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40324c0 tx=0x7f0ca4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.577+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4041020 con 0x7f0cb41016d0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.577+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb419f890 con 0x7f0cb41016d0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.577+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0ca4034cd0 con 0x7f0cb41016d0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.577+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca4039040 con 0x7f0cb41016d0 2024-03-21T08:56:57.575 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.577+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb419fd80 con 0x7f0cb41016d0 2024-03-21T08:56:57.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.578+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb4108ac0 con 0x7f0cb41016d0 2024-03-21T08:56:57.576 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.579+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0ca4044020 con 0x7f0cb41016d0 2024-03-21T08:56:57.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.579+0000 7f0c9bfff640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c9c0a2f20 0x7f0c9c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.579+0000 7f0cb27fc640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c9c0a2f20 0x7f0c9c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.579+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0ca40f11d0 con 0x7f0cb41016d0 2024-03-21T08:56:57.577 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.580+0000 7f0cb27fc640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c9c0a2f20 0x7f0c9c0a53e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4198ef0 tx=0x7f0ca8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:57.579 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.584+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0ca404b020 con 0x7f0cb41016d0 2024-03-21T08:56:57.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.797+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0cb402d070 con 0x7f0cb41016d0 2024-03-21T08:56:57.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.797+0000 7f0c9bfff640 1 -- 172.21.15.87:0/1309258191 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11460 (secure 0 0 0) 0x7f0ca40be0c0 con 0x7f0cb41016d0 2024-03-21T08:56:57.793 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:57.794 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:55:12.955598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24451,24482,24425,24487,24403]},"id":1}]} 2024-03-21T08:56:57.794 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c9c0a2f20 msgr2=0x7f0c9c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c9c0a2f20 0x7f0c9c0a53e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4198ef0 tx=0x7f0ca8005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 msgr2=0x7f0cb41979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb41979e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0ca40324c0 tx=0x7f0ca4032f40 comp rx=0 tx=0).stop 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 shutdown_connections 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0c9c0a2f20 0x7f0c9c0a53e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0cb4103280 0x7f0cb419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0cb41028c0 0x7f0cb4197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.799+0000 7f0cb97f3640 1 --2- 172.21.15.87:0/1309258191 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0cb41016d0 0x7f0cb41979e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.800+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 >> 172.21.15.87:0/1309258191 conn(0x7f0cb40fcc80 msgr2=0x7f0cb4105c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:57.796 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.800+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 shutdown_connections 2024-03-21T08:56:57.797 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.800+0000 7f0cb97f3640 1 -- 172.21.15.87:0/1309258191 wait complete. 2024-03-21T08:56:57.807 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:56:57.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 -- 172.21.15.87:0/1251688183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b0107400 msgr2=0x7f06b01097f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 --2- 172.21.15.87:0/1251688183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b0107400 0x7f06b01097f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f06a8009900 tx=0x7f06a802bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 -- 172.21.15.87:0/1251688183 shutdown_connections 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 --2- 172.21.15.87:0/1251688183 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b0107400 0x7f06b01097f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 --2- 172.21.15.87:0/1251688183 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b0104ad0 0x7f06b0106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 --2- 172.21.15.87:0/1251688183 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06b01021a0 0x7f06b0104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 -- 172.21.15.87:0/1251688183 >> 172.21.15.87:0/1251688183 conn(0x7f06b00fbb30 msgr2=0x7f06b00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.912+0000 7f06b620b640 1 -- 172.21.15.87:0/1251688183 shutdown_connections 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 -- 172.21.15.87:0/1251688183 wait complete. 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 Processor -- start 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 -- start start 2024-03-21T08:56:57.911 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 0x7f06b01979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b0104ad0 0x7f06b0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06b0107400 0x7f06b019ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b010d360 con 0x7f06b01021a0 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b010d1e0 con 0x7f06b0104ad0 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06b620b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b010d4e0 con 0x7f06b0107400 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06af7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 0x7f06b01979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06af7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 0x7f06b01979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58864/0 (socket says 172.21.15.87:58864) 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.913+0000 7f06af7fe640 1 -- 172.21.15.87:0/3155406991 learned_addr learned my addr 172.21.15.87:0/3155406991 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06affff640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06b0107400 0x7f06b019ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06aeffd640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b0104ad0 0x7f06b0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06af7fe640 1 -- 172.21.15.87:0/3155406991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06b0107400 msgr2=0x7f06b019ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06af7fe640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06b0107400 0x7f06b019ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06af7fe640 1 -- 172.21.15.87:0/3155406991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b0104ad0 msgr2=0x7f06b0197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06af7fe640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b0104ad0 0x7f06b0197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06af7fe640 1 -- 172.21.15.87:0/3155406991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b019f5f0 con 0x7f06b01021a0 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06af7fe640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 0x7f06b01979d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06a000eb90 tx=0x7f06a000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004d20 con 0x7f06b01021a0 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06b019f8e0 con 0x7f06b01021a0 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f06a000ee90 con 0x7f06b01021a0 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0010830 con 0x7f06b01021a0 2024-03-21T08:56:57.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.914+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b019fde0 con 0x7f06b01021a0 2024-03-21T08:56:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.915+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06b00fd620 con 0x7f06b01021a0 2024-03-21T08:56:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.916+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f06a0005020 con 0x7f06b01021a0 2024-03-21T08:56:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.917+0000 7f06acff9640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f06980a2f20 0x7f06980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.917+0000 7f06aeffd640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f06980a2f20 0x7f06980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.917+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f06a00cf9d0 con 0x7f06b01021a0 2024-03-21T08:56:57.914 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.917+0000 7f06aeffd640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f06980a2f20 0x7f06980a53e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06b0198ee0 tx=0x7f06a4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:57.917 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:57.921+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f06a009c830 con 0x7f06b01021a0 2024-03-21T08:56:58.096 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.100+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f06b00622a0 con 0x7f06b01021a0 2024-03-21T08:56:58.097 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.102+0000 7f06acff9640 1 -- 172.21.15.87:0/3155406991 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7f06a009c130 con 0x7f06b01021a0 2024-03-21T08:56:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:58.098 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f06980a2f20 msgr2=0x7f06980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f06980a2f20 0x7f06980a53e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f06b0198ee0 tx=0x7f06a4005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 msgr2=0x7f06b01979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 0x7f06b01979d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f06a000eb90 tx=0x7f06a000c6a0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 shutdown_connections 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f06980a2f20 0x7f06980a53e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f06b0107400 0x7f06b019ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f06b0104ad0 0x7f06b0197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 --2- 172.21.15.87:0/3155406991 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f06b01021a0 0x7f06b01979d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 >> 172.21.15.87:0/3155406991 conn(0x7f06b00fbb30 msgr2=0x7f06b0102ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:58.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 shutdown_connections 2024-03-21T08:56:58.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.104+0000 7f06b620b640 1 -- 172.21.15.87:0/3155406991 wait complete. 2024-03-21T08:56:58.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:58 smithi087 ceph-mon[32385]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:58.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:58 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1381874280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:58.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:58 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2880135694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:58.110 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:58 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1309258191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:58.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:55:04.486296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:55:05.756309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T08:56:58.112 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T08:56:58.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 -- 172.21.15.87:0/1789902358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318104a60 msgr2=0x7f9318106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 --2- 172.21.15.87:0/1789902358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318104a60 0x7f9318106e50 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f930c009900 tx=0x7f930c02bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:58.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 -- 172.21.15.87:0/1789902358 shutdown_connections 2024-03-21T08:56:58.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 --2- 172.21.15.87:0/1789902358 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318107390 0x7f93181097e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 --2- 172.21.15.87:0/1789902358 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318104a60 0x7f9318106e50 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 --2- 172.21.15.87:0/1789902358 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9318102130 0x7f9318104520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 -- 172.21.15.87:0/1789902358 >> 172.21.15.87:0/1789902358 conn(0x7f93180fbb40 msgr2=0x7f93180fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 -- 172.21.15.87:0/1789902358 shutdown_connections 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 -- 172.21.15.87:0/1789902358 wait complete. 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.220+0000 7f931dad1640 1 Processor -- start 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 -- start start 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 0x7f93181979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318104a60 0x7f9318197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9318107390 0x7f931819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931810d340 con 0x7f9318102130 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931810d1c0 con 0x7f9318104a60 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f931810d4c0 con 0x7f9318107390 2024-03-21T08:56:58.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 0x7f93181979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 0x7f93181979f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58874/0 (socket says 172.21.15.87:58874) 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 -- 172.21.15.87:0/3393367502 learned_addr learned my addr 172.21.15.87:0/3393367502 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f9317fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9318107390 0x7f931819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f9316ffd640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318104a60 0x7f9318197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 -- 172.21.15.87:0/3393367502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9318107390 msgr2=0x7f931819efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9318107390 0x7f931819efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 -- 172.21.15.87:0/3393367502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318104a60 msgr2=0x7f9318197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318104a60 0x7f9318197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 -- 172.21.15.87:0/3393367502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f931819f610 con 0x7f9318102130 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f93177fe640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 0x7f93181979f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f930800e9f0 tx=0x7f930800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f930800cd80 con 0x7f9318102130 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f9308005500 con 0x7f9318102130 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.221+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f931819f900 con 0x7f9318102130 2024-03-21T08:56:58.220 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.222+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9308010690 con 0x7f9318102130 2024-03-21T08:56:58.221 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.222+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f931819fdd0 con 0x7f9318102130 2024-03-21T08:56:58.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.223+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93180fd630 con 0x7f9318102130 2024-03-21T08:56:58.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.224+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f93080046c0 con 0x7f9318102130 2024-03-21T08:56:58.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.224+0000 7f9314ff9640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f93000a2f20 0x7f93000a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.224+0000 7f9316ffd640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f93000a2f20 0x7f93000a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.224+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f93080cf7b0 con 0x7f9318102130 2024-03-21T08:56:58.222 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.225+0000 7f9316ffd640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f93000a2f20 0x7f93000a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:58.225 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.229+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f930809c610 con 0x7f9318102130 2024-03-21T08:56:58.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:58 smithi100 ceph-mon[36306]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:58.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:58 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1381874280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:58.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:58 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2880135694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:58.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:58 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1309258191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:58.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.404+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f93180622a0 con 0x7f9318102130 2024-03-21T08:56:58.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.404+0000 7f9314ff9640 1 -- 172.21.15.87:0/3393367502 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f930809bf10 con 0x7f9318102130 2024-03-21T08:56:58.400 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f93000a2f20 msgr2=0x7f93000a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f93000a2f20 0x7f93000a53e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f930c000c00 tx=0x7f930c005fd0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 msgr2=0x7f93181979f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 0x7f93181979f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f930800e9f0 tx=0x7f930800edf0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 shutdown_connections 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f93000a2f20 0x7f93000a53e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f9318107390 0x7f931819efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f9318104a60 0x7f9318197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 --2- 172.21.15.87:0/3393367502 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f9318102130 0x7f93181979f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 >> 172.21.15.87:0/3393367502 conn(0x7f93180fbb40 msgr2=0x7f9318102bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 shutdown_connections 2024-03-21T08:56:58.403 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.406+0000 7f931dad1640 1 -- 172.21.15.87:0/3393367502 wait complete. 2024-03-21T08:56:58.413 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T08:56:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:58 smithi105 ceph-mon[35993]: pgmap v347: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:56:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:58 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1381874280' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:58 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2880135694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:56:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:58 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1309258191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:56:58.519 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 -- 172.21.15.87:0/2123495879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 msgr2=0x7f33e8101ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 --2- 172.21.15.87:0/2123495879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8101ad0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f33e4009ca0 tx=0x7f33e402c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 -- 172.21.15.87:0/2123495879 shutdown_connections 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 --2- 172.21.15.87:0/2123495879 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e8103260 0x7f33e8109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 --2- 172.21.15.87:0/2123495879 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e81028a0 0x7f33e8102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 --2- 172.21.15.87:0/2123495879 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8101ad0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 -- 172.21.15.87:0/2123495879 >> 172.21.15.87:0/2123495879 conn(0x7f33e80fcc40 msgr2=0x7f33e80ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 -- 172.21.15.87:0/2123495879 shutdown_connections 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.521+0000 7f33efe59640 1 -- 172.21.15.87:0/2123495879 wait complete. 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 Processor -- start 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 -- start start 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e81028a0 0x7f33e8197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.520 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e8103260 0x7f33e819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e810d330 con 0x7f33e81016b0 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e810d1b0 con 0x7f33e81028a0 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33efe59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e810d4b0 con 0x7f33e8103260 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8197a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58896/0 (socket says 172.21.15.87:58896) 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33ed3f4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e81028a0 0x7f33e8197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 -- 172.21.15.87:0/1164758711 learned_addr learned my addr 172.21.15.87:0/1164758711 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33ee3f6640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e8103260 0x7f33e819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 -- 172.21.15.87:0/1164758711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e8103260 msgr2=0x7f33e819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e8103260 0x7f33e819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 -- 172.21.15.87:0/1164758711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e81028a0 msgr2=0x7f33e8197f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e81028a0 0x7f33e8197f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.521 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33edbf5640 1 -- 172.21.15.87:0/1164758711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e819f670 con 0x7f33e81016b0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.522+0000 7f33ed3f4640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e81028a0 0x7f33e8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.523+0000 7f33edbf5640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8197a50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f33e40324c0 tx=0x7f33e4032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.523+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4041020 con 0x7f33e81016b0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.523+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e819f900 con 0x7f33e81016b0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.523+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33e4034cd0 con 0x7f33e81016b0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.523+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e4039040 con 0x7f33e81016b0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.523+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e819fdf0 con 0x7f33e81016b0 2024-03-21T08:56:58.522 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.524+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8108af0 con 0x7f33e81016b0 2024-03-21T08:56:58.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.525+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f33e4044020 con 0x7f33e81016b0 2024-03-21T08:56:58.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.526+0000 7f33deffd640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33d40a2f20 0x7f33d40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.526+0000 7f33ed3f4640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33d40a2f20 0x7f33d40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.526+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f33e40f1630 con 0x7f33e81016b0 2024-03-21T08:56:58.523 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.526+0000 7f33ed3f4640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33d40a2f20 0x7f33d40a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33e8198f60 tx=0x7f33d8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:58.525 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.529+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f33e404b020 con 0x7f33e81016b0 2024-03-21T08:56:58.699 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.703+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33e802d070 con 0x7f33e81016b0 2024-03-21T08:56:58.700 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.704+0000 7f33deffd640 1 -- 172.21.15.87:0/1164758711 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33e40be520 con 0x7f33e81016b0 2024-03-21T08:56:58.700 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-03-21T08:56:58.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33d40a2f20 msgr2=0x7f33d40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33d40a2f20 0x7f33d40a53e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f33e8198f60 tx=0x7f33d8005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:58.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 msgr2=0x7f33e8197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8197a50 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f33e40324c0 tx=0x7f33e4032f40 comp rx=0 tx=0).stop 2024-03-21T08:56:58.702 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 shutdown_connections 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33d40a2f20 0x7f33d40a53e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33e8103260 0x7f33e819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33e81028a0 0x7f33e8197f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 --2- 172.21.15.87:0/1164758711 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33e81016b0 0x7f33e8197a50 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 >> 172.21.15.87:0/1164758711 conn(0x7f33e80fcc40 msgr2=0x7f33e8105cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 shutdown_connections 2024-03-21T08:56:58.703 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.706+0000 7f33efe59640 1 -- 172.21.15.87:0/1164758711 wait complete. 2024-03-21T08:56:58.713 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T08:56:58.818 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 -- 172.21.15.87:0/2368783981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 msgr2=0x7f40cc1089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2368783981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc1089e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f40c4009900 tx=0x7f40c402bc30 comp rx=0 tx=0).stop 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 -- 172.21.15.87:0/2368783981 shutdown_connections 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2368783981 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc1089e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2368783981 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40cc101790 0x7f40cc101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2368783981 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 0x7f40cc1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 -- 172.21.15.87:0/2368783981 >> 172.21.15.87:0/2368783981 conn(0x7f40cc0fbb30 msgr2=0x7f40cc0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 -- 172.21.15.87:0/2368783981 shutdown_connections 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.820+0000 7f40d0d68640 1 -- 172.21.15.87:0/2368783981 wait complete. 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 Processor -- start 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 -- start start 2024-03-21T08:56:58.819 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 0x7f40cc069370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40cc101790 0x7f40cc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc10c200 con 0x7f40cc102150 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc10c080 con 0x7f40cc1005a0 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40d0d68640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40cc10c380 con 0x7f40cc101790 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40cad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40cad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc19ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58904/0 (socket says 172.21.15.87:58904) 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40ca59c640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 0x7f40cc069370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40cad9d640 1 -- 172.21.15.87:0/2415551941 learned_addr learned my addr 172.21.15.87:0/2415551941 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.821+0000 7f40c9d9b640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40cc101790 0x7f40cc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40cad9d640 1 -- 172.21.15.87:0/2415551941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40cc101790 msgr2=0x7f40cc19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40cad9d640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40cc101790 0x7f40cc19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.820 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40cad9d640 1 -- 172.21.15.87:0/2415551941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 msgr2=0x7f40cc069370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40cad9d640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 0x7f40cc069370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40cad9d640 1 -- 172.21.15.87:0/2415551941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc19f620 con 0x7f40cc102150 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40ca59c640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 0x7f40cc069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40cad9d640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc19ed10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005ab0 tx=0x7f40c402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4041070 con 0x7f40cc102150 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f40c4004450 con 0x7f40cc102150 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40cc19f850 con 0x7f40cc102150 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c4003e60 con 0x7f40cc102150 2024-03-21T08:56:58.821 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.822+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40cc19fd40 con 0x7f40cc102150 2024-03-21T08:56:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.823+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40cc068490 con 0x7f40cc102150 2024-03-21T08:56:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.824+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f40c4044070 con 0x7f40cc102150 2024-03-21T08:56:58.822 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.824+0000 7f40bb7fe640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40b40a2f20 0x7f40b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.824+0000 7f40ca59c640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40b40a2f20 0x7f40b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.825+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f40c40f0ff0 con 0x7f40cc102150 2024-03-21T08:56:58.823 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.825+0000 7f40ca59c640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40b40a2f20 0x7f40b40a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0099a0 tx=0x7f40bc009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:58.824 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:58.829+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f40c40bd800 con 0x7f40cc102150 2024-03-21T08:56:59.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.008+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f40cc02d070 con 0x7f40cc102150 2024-03-21T08:56:59.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.009+0000 7f40bb7fe640 1 -- 172.21.15.87:0/2415551941 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f40c40c18a0 con 0x7f40cc102150 2024-03-21T08:56:59.005 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40b40a2f20 msgr2=0x7f40b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40b40a2f20 0x7f40b40a53e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f40bc0099a0 tx=0x7f40bc009500 comp rx=0 tx=0).stop 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 msgr2=0x7f40cc19ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc19ed10 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f40c4005ab0 tx=0x7f40c402c6c0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 shutdown_connections 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f40b40a2f20 0x7f40b40a53e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f40cc102150 0x7f40cc19ed10 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f40cc101790 0x7f40cc19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 --2- 172.21.15.87:0/2415551941 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f40cc1005a0 0x7f40cc069370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 >> 172.21.15.87:0/2415551941 conn(0x7f40cc0fbb30 msgr2=0x7f40cc104a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 shutdown_connections 2024-03-21T08:56:59.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.011+0000 7f40d0d68640 1 -- 172.21.15.87:0/2415551941 wait complete. 2024-03-21T08:56:59.018 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 -- 172.21.15.87:0/2047678265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 msgr2=0x7f676c1009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 --2- 172.21.15.87:0/2047678265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c1009e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f675c009ca0 tx=0x7f675c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 -- 172.21.15.87:0/2047678265 shutdown_connections 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 --2- 172.21.15.87:0/2047678265 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f676c102170 0x7f676c108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 --2- 172.21.15.87:0/2047678265 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 0x7f676c101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 --2- 172.21.15.87:0/2047678265 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c1009e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 -- 172.21.15.87:0/2047678265 >> 172.21.15.87:0/2047678265 conn(0x7f676c0fbb30 msgr2=0x7f676c0fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 -- 172.21.15.87:0/2047678265 shutdown_connections 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.131+0000 7f6770d6f640 1 -- 172.21.15.87:0/2047678265 wait complete. 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 Processor -- start 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 -- start start 2024-03-21T08:56:59.130 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c069390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 0x7f676c19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f676c102170 0x7f676c19ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c10c230 con 0x7f676c1005c0 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c10c0b0 con 0x7f676c102170 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6770d6f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c10c3b0 con 0x7f676c1017b0 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f676a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c069390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f6769d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 0x7f676c19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f676a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c069390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58930/0 (socket says 172.21.15.87:58930) 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f676a59c640 1 -- 172.21.15.87:0/1381819186 learned_addr learned my addr 172.21.15.87:0/1381819186 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.132+0000 7f676ad9d640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f676c102170 0x7f676c19ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6769d9b640 1 -- 172.21.15.87:0/1381819186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f676c102170 msgr2=0x7f676c19ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.131 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6769d9b640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f676c102170 0x7f676c19ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6769d9b640 1 -- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 msgr2=0x7f676c069390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6769d9b640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c069390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6769d9b640 1 -- 172.21.15.87:0/1381819186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c19f630 con 0x7f676c1017b0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f676a59c640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6769d9b640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 0x7f676c19c730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f676000e9f0 tx=0x7f676000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676000ccc0 con 0x7f676c1017b0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c19f920 con 0x7f676c1017b0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f6760005020 con 0x7f676c1017b0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c19fe70 con 0x7f676c1017b0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.133+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67600049e0 con 0x7f676c1017b0 2024-03-21T08:56:59.132 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.134+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c107960 con 0x7f676c1017b0 2024-03-21T08:56:59.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.135+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f6760005560 con 0x7f676c1017b0 2024-03-21T08:56:59.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.136+0000 7f675b7fe640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67540a2f20 0x7f67540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.136+0000 7f676a59c640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67540a2f20 0x7f67540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.133 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.136+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f67600cf3c0 con 0x7f676c1017b0 2024-03-21T08:56:59.134 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.136+0000 7f676a59c640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67540a2f20 0x7f67540a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f675c032420 tx=0x7f675c032590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.145+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f67600051d0 con 0x7f676c1017b0 2024-03-21T08:56:59.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:59 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3155406991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:59.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:59 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3393367502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:59.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:56:59 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1164758711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:59.353 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.357+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f676c02d220 con 0x7f676c1017b0 2024-03-21T08:56:59.354 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.358+0000 7f675b7fe640 1 -- 172.21.15.87:0/1381819186 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+11460 (secure 0 0 0) 0x7f676009c290 con 0x7f676c1017b0 2024-03-21T08:56:59.354 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:59.354 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:55:12.955598+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24451,24482,24425,24487,24403]},"id":1}]} 2024-03-21T08:56:59.355 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 25 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67540a2f20 msgr2=0x7f67540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67540a2f20 0x7f67540a53e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f675c032420 tx=0x7f675c032590 comp rx=0 tx=0).stop 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 msgr2=0x7f676c19c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 0x7f676c19c730 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f676000e9f0 tx=0x7f676000eeb0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 shutdown_connections 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f67540a2f20 0x7f67540a53e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f676c102170 0x7f676c19ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f676c1017b0 0x7f676c19c730 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 --2- 172.21.15.87:0/1381819186 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f676c1005c0 0x7f676c069390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.357 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 >> 172.21.15.87:0/1381819186 conn(0x7f676c0fbb30 msgr2=0x7f676c104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 shutdown_connections 2024-03-21T08:56:59.358 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.361+0000 7f6770d6f640 1 -- 172.21.15.87:0/1381819186 wait complete. 2024-03-21T08:56:59.366 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:59 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3155406991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:59.366 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:59 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3393367502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:59.366 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:56:59 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1164758711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:59.368 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T08:56:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 -- 172.21.15.87:0/1314077677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 msgr2=0x7fbab0101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 --2- 172.21.15.87:0/1314077677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0101b00 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fbaac009ca0 tx=0x7fbaac02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.472 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 -- 172.21.15.87:0/1314077677 shutdown_connections 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 --2- 172.21.15.87:0/1314077677 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab0103290 0x7fbab0109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 --2- 172.21.15.87:0/1314077677 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab01028d0 0x7fbab0102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 --2- 172.21.15.87:0/1314077677 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0101b00 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.473+0000 7fbab867e640 1 -- 172.21.15.87:0/1314077677 >> 172.21.15.87:0/1314077677 conn(0x7fbab00fcc50 msgr2=0x7fbab00ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 -- 172.21.15.87:0/1314077677 shutdown_connections 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 -- 172.21.15.87:0/1314077677 wait complete. 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 Processor -- start 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 -- start start 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab01028d0 0x7fbab0073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.473 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab0103290 0x7fbab0073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab010d330 con 0x7fbab01016e0 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab010d1b0 con 0x7fbab0103290 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab867e640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbab010d4b0 con 0x7fbab01028d0 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab641a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab641a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0075450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58944/0 (socket says 172.21.15.87:58944) 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab641a640 1 -- 172.21.15.87:0/2019517876 learned_addr learned my addr 172.21.15.87:0/2019517876 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab5c19640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab01028d0 0x7fbab0073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.474+0000 7fbab6c1b640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab0103290 0x7fbab0073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab641a640 1 -- 172.21.15.87:0/2019517876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab01028d0 msgr2=0x7fbab0073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab641a640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab01028d0 0x7fbab0073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab641a640 1 -- 172.21.15.87:0/2019517876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab0103290 msgr2=0x7fbab0073fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab641a640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab0103290 0x7fbab0073fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.474 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab641a640 1 -- 172.21.15.87:0/2019517876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbab0074860 con 0x7fbab01016e0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab6c1b640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab0103290 0x7fbab0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab641a640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0075450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0324c0 tx=0x7fbaac032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac041020 con 0x7fbab01016e0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbab01a1ac0 con 0x7fbab01016e0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbaac034cd0 con 0x7fbab01016e0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaac039040 con 0x7fbab01016e0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.475+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbab01a2060 con 0x7fbab01016e0 2024-03-21T08:56:59.475 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.476+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbab0108a80 con 0x7fbab01016e0 2024-03-21T08:56:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.477+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbaac044020 con 0x7fbab01016e0 2024-03-21T08:56:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.477+0000 7fbaa77fe640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fba9c0a2f20 0x7fba9c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.478+0000 7fbab5c19640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fba9c0a2f20 0x7fba9c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.478+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbaac0f1700 con 0x7fbab01016e0 2024-03-21T08:56:59.476 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.478+0000 7fbab5c19640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fba9c0a2f20 0x7fba9c0a53e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbab00751c0 tx=0x7fbaa0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.477 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.481+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbaac04b020 con 0x7fbab01016e0 2024-03-21T08:56:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:59 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3155406991' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:56:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:59 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3393367502' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T08:56:59.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:56:59 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1164758711' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T08:56:59.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.663+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbab002d270 con 0x7fbab01016e0 2024-03-21T08:56:59.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.665+0000 7fbaa77fe640 1 -- 172.21.15.87:0/2019517876 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7fbaac0be5f0 con 0x7fbab01016e0 2024-03-21T08:56:59.661 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T08:56:59.661 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fba9c0a2f20 msgr2=0x7fba9c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fba9c0a2f20 0x7fba9c0a53e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fbab00751c0 tx=0x7fbaa0005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 msgr2=0x7fbab0075450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0075450 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fbaac0324c0 tx=0x7fbaac032f40 comp rx=0 tx=0).stop 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 shutdown_connections 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fba9c0a2f20 0x7fba9c0a53e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbab0103290 0x7fbab0073fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbab01028d0 0x7fbab0073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 --2- 172.21.15.87:0/2019517876 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbab01016e0 0x7fbab0075450 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 >> 172.21.15.87:0/2019517876 conn(0x7fbab00fcc50 msgr2=0x7fbab0105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 shutdown_connections 2024-03-21T08:56:59.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.667+0000 7fbab867e640 1 -- 172.21.15.87:0/2019517876 wait complete. 2024-03-21T08:56:59.674 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-03-21T08:55:06.567291+0000', 'modified': '2024-03-21T08:55:12.955598+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24487, 'mds_1': 24482, 'mds_2': 24451, 'mds_3': 24425, 'mds_4': 24403}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14634': {'gid': 14634, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6839/3694307942', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6838', 'nonce': 3694307942}, {'type': 'v1', 'addr': '172.21.15.87:6839', 'nonce': 3694307942}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_14646': {'gid': 14646, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.87:6841/2007269086', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.87:6840', 'nonce': 2007269086}, {'type': 'v1', 'addr': '172.21.15.87:6841', 'nonce': 2007269086}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24395': {'gid': 24395, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6833/1637520984', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6832', 'nonce': 1637520984}, {'type': 'v1', 'addr': '172.21.15.100:6833', 'nonce': 1637520984}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24403': {'gid': 24403, 'name': 'l', 'rank': 4, 'incarnation': 23, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.105:6833/500134057', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6832', 'nonce': 500134057}, {'type': 'v1', 'addr': '172.21.15.105:6833', 'nonce': 500134057}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24425': {'gid': 24425, 'name': 'e', 'rank': 3, 'incarnation': 21, 'state': 'up:active', 'state_seq': 9, 'addr': '172.21.15.100:6835/2548903221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6834', 'nonce': 2548903221}, {'type': 'v1', 'addr': '172.21.15.100:6835', 'nonce': 2548903221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24451': {'gid': 24451, 'name': 'f', 'rank': 2, 'incarnation': 19, 'state': 'up:active', 'state_seq': 8, 'addr': '172.21.15.105:6835/1031026542', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6834', 'nonce': 1031026542}, {'type': 'v1', 'addr': '172.21.15.105:6835', 'nonce': 1031026542}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.100:6837/2948361419', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6836', 'nonce': 2948361419}, {'type': 'v1', 'addr': '172.21.15.100:6837', 'nonce': 2948361419}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24469': {'gid': 24469, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.105:6837/2656551527', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6836', 'nonce': 2656551527}, {'type': 'v1', 'addr': '172.21.15.105:6837', 'nonce': 2656551527}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 1, 'incarnation': 17, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.100:6839/214101858', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.100:6838', 'nonce': 214101858}, {'type': 'v1', 'addr': '172.21.15.100:6839', 'nonce': 214101858}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_24487': {'gid': 24487, 'name': 'i', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.105:6839/892092628', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.105:6838', 'nonce': 892092628}, {'type': 'v1', 'addr': '172.21.15.105:6839', 'nonce': 892092628}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24487, 'qdb_cluster': [24451, 24482, 24425, 24487, 24403]} 2024-03-21T08:56:59.674 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-03-21T08:56:59.674 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-03-21T08:56:59.778 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.780+0000 7f7fb156f640 1 -- 172.21.15.87:0/2597878664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 msgr2=0x7f7fac1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.780+0000 7f7fb156f640 1 --2- 172.21.15.87:0/2597878664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac1089f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c009ca0 tx=0x7f7f9c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- 172.21.15.87:0/2597878664 shutdown_connections 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 --2- 172.21.15.87:0/2597878664 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac1089f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 --2- 172.21.15.87:0/2597878664 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 --2- 172.21.15.87:0/2597878664 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 0x7f7fac1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- 172.21.15.87:0/2597878664 >> 172.21.15.87:0/2597878664 conn(0x7f7fac0fbb40 msgr2=0x7f7fac0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- 172.21.15.87:0/2597878664 shutdown_connections 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- 172.21.15.87:0/2597878664 wait complete. 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 Processor -- start 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- start start 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 0x7f7fac06a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.779 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac19ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10c230 con 0x7f7fac102160 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10c0b0 con 0x7f7fac1005b0 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.781+0000 7f7fb156f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fac10c3b0 con 0x7f7fac1017a0 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac19ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7faa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac19ece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58966/0 (socket says 172.21.15.87:58966) 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7faa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac19c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:51180/0 (socket says 172.21.15.87:51180) 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 -- 172.21.15.87:0/3534941091 learned_addr learned my addr 172.21.15.87:0/3534941091 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7faaffd640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 0x7f7fac06a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 -- 172.21.15.87:0/3534941091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 msgr2=0x7f7fac19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.780 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 -- 172.21.15.87:0/3534941091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 msgr2=0x7f7fac06a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 0x7f7fac06a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 -- 172.21.15.87:0/3534941091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fac19f5a0 con 0x7f7fac102160 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7faaffd640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 0x7f7fac06a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac19ece0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c037040 tx=0x7f7f9c032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fa3fff640 1 -- 172.21.15.87:0/3534941091 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c041020 con 0x7f7fac102160 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fac19f830 con 0x7f7fac102160 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fa3fff640 1 -- 172.21.15.87:0/3534941091 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f9c034cd0 con 0x7f7fac102160 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fa3fff640 1 -- 172.21.15.87:0/3534941091 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c039040 con 0x7f7fac102160 2024-03-21T08:56:59.781 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.782+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fac19fd20 con 0x7f7fac102160 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.784+0000 7f7fa3fff640 1 -- 172.21.15.87:0/3534941091 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f9c044020 con 0x7f7fac102160 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 Processor -- start 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fa3fff640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f980a2f20 0x7f7f980a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 -- start start 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f980a2f20 0x7f7f980a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fa3fff640 1 -- 172.21.15.87:0/3534941091 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7f9c0f1280 con 0x7f7fac102160 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b3b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9bb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.788 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e918c20 con 0x55fc4e9b3740 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e8c1a40 con 0x55fc4e9b92a0 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fb4231740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e914cc0 con 0x55fc4e9b4130 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faa7fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b3b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58976/0 (socket says 172.21.15.87:58976) 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 -- 172.21.15.87:0/4030341916 learned_addr learned my addr 172.21.15.87:0/4030341916 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9bb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f980a2f20 0x7f7f980a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fac06a570 tx=0x7f7f94005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 -- 172.21.15.87:0/4030341916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 msgr2=0x55fc4e9b8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 -- 172.21.15.87:0/4030341916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 msgr2=0x55fc4e9bb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9bb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.789 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7faaffd640 1 -- 172.21.15.87:0/4030341916 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4e8e3f90 con 0x55fc4e9b3740 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.785+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7faaffd640 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b3b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40114f0 tx=0x7f7fa40118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db79b8553a867ad5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fa17fa640 1 -- 172.21.15.87:0/4030341916 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401e070 con 0x55fc4e9b3740 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fa17fa640 1 -- 172.21.15.87:0/4030341916 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fa400f9c0 con 0x55fc4e9b3740 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fa17fa640 1 -- 172.21.15.87:0/4030341916 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4019a90 con 0x55fc4e9b3740 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fb4231740 1 -- 172.21.15.87:0/4030341916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 msgr2=0x55fc4e9b3b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fb4231740 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b3b60 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40114f0 tx=0x7f7fa40118f0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fb4231740 1 -- 172.21.15.87:0/4030341916 shutdown_connections 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.786+0000 7f7fb4231740 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9bb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 --2- 172.21.15.87:0/4030341916 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b3b60 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.790 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- 172.21.15.87:0/4030341916 >> 172.21.15.87:0/4030341916 conn(0x55fc4e97ba60 msgr2=0x55fc4e9af0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- 172.21.15.87:0/4030341916 shutdown_connections 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- 172.21.15.87:0/4030341916 wait complete. 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- start start 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 Processor -- start 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b3740 0x55fc4e9bdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9be1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9bc250 con 0x55fc4e9b92a0 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e8d4e90 con 0x55fc4e9b4130 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9bc3d0 con 0x55fc4e9b3740 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fb4231740 10 client.0 _ll_register_callbacks cb 0x55fc4e62cac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58982/0 (socket says 172.21.15.87:58982) 2024-03-21T08:56:59.791 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b3740 0x55fc4e9bdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 -- 172.21.15.87:0/1420018645 learned_addr learned my addr 172.21.15.87:0/1420018645 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9be1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 -- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b3740 msgr2=0x55fc4e9bdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b3740 0x55fc4e9bdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 -- 172.21.15.87:0/1420018645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 msgr2=0x55fc4e9be1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9be1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.787+0000 7f7fab7fe640 1 -- 172.21.15.87:0/1420018645 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc4e8e19d0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0b9ed0 tx=0x7f7f9c0bd3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b748ccf03bed1837 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7f927fc640 1 -- 172.21.15.87:0/1420018645 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0c1bc0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7f927fc640 1 -- 172.21.15.87:0/1420018645 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7f9c003970 con 0x55fc4e9b92a0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fa400f9c0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7f927fc640 1 -- 172.21.15.87:0/1420018645 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f9c0c63f0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 10 client.15270 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:59.792 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fc4e62d2d0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7f927fc640 1 -- 172.21.15.87:0/1420018645 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fa400f9c0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7f927fc640 1 -- 172.21.15.87:0/1420018645 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7f9c03c030 con 0x55fc4e9b92a0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 10 client.15270 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 10 client.15270 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 20 client.15270 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4e9ca3f0 0x55fc4e9c9340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 4 client.15270 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 --> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc4e926160 con 0x55fc4e9ca3f0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.788+0000 7f7faaffd640 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4e9ca3f0 0x55fc4e9c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.789+0000 7f7faaffd640 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4e9ca3f0 0x55fc4e9c9340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=231c41f11bea09fa server_cookie=ee01440d72345aac in_seq=0 out_seq=0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.789+0000 7f7f927fc640 10 client.15270 ms_handle_connect on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.789+0000 7f7f927fc640 1 -- 172.21.15.87:0/1420018645 <== mds.4 v2:172.21.15.105:6832/500134057 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55fc4e926160 con 0x55fc4e9ca3f0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.789+0000 7f7f927fc640 10 client.15270 handle_command_reply: tid=0 2024-03-21T08:56:59.793 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.789+0000 7f7fb4231740 1 client.15270 shutdown 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 msgr2=0x55fc4e9c51e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c0b9ed0 tx=0x7f7f9c0bd3d0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 shutdown_connections 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4e9ca3f0 0x55fc4e9c9340 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9be1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7f927fc640 0 client.15270 ms_handle_reset on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 --2- 172.21.15.87:0/1420018645 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b3740 0x55fc4e9bdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 >> 172.21.15.87:0/1420018645 conn(0x55fc4e97ba60 msgr2=0x55fc4e9ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 shutdown_connections 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 1 -- 172.21.15.87:0/1420018645 wait complete. 2024-03-21T08:56:59.794 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.790+0000 7f7fb4231740 20 client.15270 trim_cache size 0 max 16384 2024-03-21T08:56:59.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 Processor -- start 2024-03-21T08:56:59.807 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 -- start start 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b55d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b5b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9c9c10 con 0x55fc4e9b3740 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9ca270 con 0x55fc4e9b92a0 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fb4231740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9ca8d0 con 0x55fc4e9b4130 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:58996/0 (socket says 172.21.15.87:58996) 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 -- 172.21.15.87:0/573311276 learned_addr learned my addr 172.21.15.87:0/573311276 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 -- 172.21.15.87:0/573311276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 msgr2=0x55fc4e9b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b5b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 -- 172.21.15.87:0/573311276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 msgr2=0x55fc4e9c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faaffd640 1 -- 172.21.15.87:0/573311276 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc4e6f76b0 con 0x55fc4e9b3740 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.806+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7faaffd640 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b55d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005fd0 tx=0x7f7fa400ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35aabfb4e375c3ab server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fa0ff9640 1 -- 172.21.15.87:0/573311276 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401e040 con 0x55fc4e9b3740 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fa0ff9640 1 -- 172.21.15.87:0/573311276 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fa4029070 con 0x55fc4e9b3740 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fa0ff9640 1 -- 172.21.15.87:0/573311276 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa400f9f0 con 0x55fc4e9b3740 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 -- 172.21.15.87:0/573311276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 msgr2=0x55fc4e9b55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b55d0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4005fd0 tx=0x7f7fa400ff40 comp rx=0 tx=0).stop 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 -- 172.21.15.87:0/573311276 shutdown_connections 2024-03-21T08:56:59.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b92a0 0x55fc4e9c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b4130 0x55fc4e9b5b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 --2- 172.21.15.87:0/573311276 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9b55d0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 -- 172.21.15.87:0/573311276 >> 172.21.15.87:0/573311276 conn(0x55fc4e97ba60 msgr2=0x55fc4e9bdc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 -- 172.21.15.87:0/573311276 shutdown_connections 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.807+0000 7f7fb4231740 1 -- 172.21.15.87:0/573311276 wait complete. 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 -- start start 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 Processor -- start 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9c1e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b92a0 0x55fc4ea25450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9cb590 con 0x55fc4e9b3740 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9caf30 con 0x55fc4e9b4130 2024-03-21T08:56:59.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc4e9cbbf0 con 0x55fc4e9b92a0 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fb4231740 10 client.0 _ll_register_callbacks cb 0x7f7f9c009900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9c1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9c1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59000/0 (socket says 172.21.15.87:59000) 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 -- 172.21.15.87:0/3526166024 learned_addr learned my addr 172.21.15.87:0/3526166024 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7fab7fe640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b92a0 0x55fc4ea25450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 -- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b92a0 msgr2=0x55fc4ea25450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b92a0 0x55fc4ea25450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 -- 172.21.15.87:0/3526166024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 msgr2=0x55fc4e9c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9c23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 -- 172.21.15.87:0/3526166024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fc4e9c5400 con 0x55fc4e9b3740 2024-03-21T08:56:59.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7faaffd640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9c1e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa40118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6645d0db4a1169c server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.808+0000 7f7f927fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7f927fc640 1 -- 172.21.15.87:0/3526166024 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa401e040 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7f927fc640 1 -- 172.21.15.87:0/3526166024 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f7fa400f730 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7fa4029070 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7f927fc640 1 -- 172.21.15.87:0/3526166024 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7fa402a2c0 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 10 client.15276 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55fc4ea269c0 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7f927fc640 1 -- 172.21.15.87:0/3526166024 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7fa4029070 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7f927fc640 1 -- 172.21.15.87:0/3526166024 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f7fa40035f0 con 0x55fc4e9b3740 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 10 client.15276 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 10 client.15276 resolve_mds: resolved name 'l' to daemon mds.l 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 20 client.15276 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:59.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4ea2aab0 0x55fc4ea26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 4 client.15276 mds_command: new command op to 24403 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 --> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fc4e9204b0 con 0x55fc4ea2aab0 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.809+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4ea2aab0 0x55fc4ea26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7faa7fc640 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4ea2aab0 0x55fc4ea26b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9834ee1615247c07 server_cookie=563af25c9710c349 in_seq=0 out_seq=0 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7f927fc640 10 client.15276 ms_handle_connect on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7f927fc640 1 -- 172.21.15.87:0/3526166024 <== mds.4 v2:172.21.15.105:6832/500134057 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x55fc4e9204b0 con 0x55fc4ea2aab0 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7f927fc640 10 client.15276 handle_command_reply: tid=0 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 client.15276 shutdown 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 msgr2=0x55fc4e9c1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9c1e90 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000c00 tx=0x7f7fa40118f0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 shutdown_connections 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:6832/500134057,v1:172.21.15.105:6833/500134057] conn(0x55fc4ea2aab0 0x55fc4ea26b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55fc4e9b92a0 0x55fc4ea25450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.813 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55fc4e9b4130 0x55fc4e9c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7f927fc640 0 client.15276 ms_handle_reset on v2:172.21.15.105:6832/500134057 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 --2- 172.21.15.87:0/3526166024 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55fc4e9b3740 0x55fc4e9c1e90 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 >> 172.21.15.87:0/3526166024 conn(0x55fc4e97ba60 msgr2=0x55fc4e9aebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 shutdown_connections 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 1 -- 172.21.15.87:0/3526166024 wait complete. 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.810+0000 7f7fb4231740 20 client.15276 trim_cache size 0 max 16384 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 4, 2024-03-21T08:56:59.814 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24403, 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 107.859911645, 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 150.753146106 2024-03-21T08:56:59.815 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f980a2f20 msgr2=0x7f7f980a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f980a2f20 0x7f7f980a53e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7fac06a570 tx=0x7f7f94005f90 comp rx=0 tx=0).stop 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 msgr2=0x7f7fac19ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac19ece0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c037040 tx=0x7f7f9c032f40 comp rx=0 tx=0).stop 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 shutdown_connections 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f980a2f20 0x7f7f980a53e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.816 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7fac102160 0x7f7fac19ece0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7fac1017a0 0x7f7fac19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 --2- 172.21.15.87:0/3534941091 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7fac1005b0 0x7f7fac06a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 >> 172.21.15.87:0/3534941091 conn(0x7f7fac0fbb40 msgr2=0x7f7fac104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 shutdown_connections 2024-03-21T08:56:59.817 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.811+0000 7f7fb156f640 1 -- 172.21.15.87:0/3534941091 wait complete. 2024-03-21T08:56:59.818 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2024-03-21T08:56:59.925 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 -- 172.21.15.87:0/3642426351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc1005b0 msgr2=0x7f33cc1009d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/3642426351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc1005b0 0x7f33cc1009d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f33bc009860 tx=0x7f33bc02bb70 comp rx=0 tx=0).stop 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 -- 172.21.15.87:0/3642426351 shutdown_connections 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/3642426351 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33cc102160 0x7f33cc1089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/3642426351 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1017a0 0x7f33cc101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/3642426351 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc1005b0 0x7f33cc1009d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.928+0000 7f33d0f3b640 1 -- 172.21.15.87:0/3642426351 >> 172.21.15.87:0/3642426351 conn(0x7f33cc0fbb40 msgr2=0x7f33cc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 -- 172.21.15.87:0/3642426351 shutdown_connections 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 -- 172.21.15.87:0/3642426351 wait complete. 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 Processor -- start 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 -- start start 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 0x7f33cc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.926 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33cc1017a0 0x7f33cc197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc102160 0x7f33cc19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc10d350 con 0x7f33cc1005b0 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc10d1d0 con 0x7f33cc102160 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33d0f3b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc10d4d0 con 0x7f33cc1017a0 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.929+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 0x7f33cc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 0x7f33cc197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59010/0 (socket says 172.21.15.87:59010) 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 -- 172.21.15.87:0/1018318735 learned_addr learned my addr 172.21.15.87:0/1018318735 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33cc1017a0 0x7f33cc197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33cad9d640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc102160 0x7f33cc19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 -- 172.21.15.87:0/1018318735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33cc1017a0 msgr2=0x7f33cc197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33cc1017a0 0x7f33cc197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.927 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 -- 172.21.15.87:0/1018318735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc102160 msgr2=0x7f33cc19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc102160 0x7f33cc19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 -- 172.21.15.87:0/1018318735 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc19f620 con 0x7f33cc1005b0 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33ca59c640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 0x7f33cc197a00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f33bc02c0b0 tx=0x7f33bc032430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33bb7fe640 1 -- 172.21.15.87:0/1018318735 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc041020 con 0x7f33cc1005b0 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33bb7fe640 1 -- 172.21.15.87:0/1018318735 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33bc02c9f0 con 0x7f33cc1005b0 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33cc19f8b0 con 0x7f33cc1005b0 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33bb7fe640 1 -- 172.21.15.87:0/1018318735 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc039040 con 0x7f33cc1005b0 2024-03-21T08:56:59.928 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.930+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33cc19fda0 con 0x7f33cc1005b0 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.932+0000 7f33bb7fe640 1 -- 172.21.15.87:0/1018318735 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f33bc044020 con 0x7f33cc1005b0 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.932+0000 7f33d3c04740 1 Processor -- start 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 -- start start 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257ee0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257ee6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ee8b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e41c20 con 0x563257ee66b0 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257deaa40 con 0x563257ee0b50 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33d3c04740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257e3dcc0 con 0x563257ee1540 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ee8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.935 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257ee6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33cad9d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257ee0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ee8b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59020/0 (socket says 172.21.15.87:59020) 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 -- 172.21.15.87:0/675825281 learned_addr learned my addr 172.21.15.87:0/675825281 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33bb7fe640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33b40a2f20 0x7f33b40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33cad9d640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33b40a2f20 0x7f33b40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 -- 172.21.15.87:0/675825281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 msgr2=0x563257ee6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257ee6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 -- 172.21.15.87:0/675825281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 msgr2=0x563257ee0f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257ee0f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 -- 172.21.15.87:0/675825281 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563257e0cf90 con 0x563257ee66b0 2024-03-21T08:56:59.936 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33bb7fe640 1 -- 172.21.15.87:0/1018318735 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f33bc0f0ca0 con 0x7f33cc1005b0 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ee8b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009c90 tx=0x7f33c0011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b616cf4e4eb8d9e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33cad9d640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33b40a2f20 0x7f33b40a53e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x563257ee1da0 tx=0x7f33c4004ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.933+0000 7f33b8ff9640 1 -- 172.21.15.87:0/675825281 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c001e070 con 0x563257ee66b0 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33b8ff9640 1 -- 172.21.15.87:0/675825281 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c0019b10 con 0x563257ee66b0 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33b8ff9640 1 -- 172.21.15.87:0/675825281 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c000ea30 con 0x563257ee66b0 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 -- 172.21.15.87:0/675825281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 msgr2=0x563257ee8b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ee8b40 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f33c0009c90 tx=0x7f33c0011810 comp rx=0 tx=0).stop 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 -- 172.21.15.87:0/675825281 shutdown_connections 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ee8b40 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257ee6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 --2- 172.21.15.87:0/675825281 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257ee0f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 -- 172.21.15.87:0/675825281 >> 172.21.15.87:0/675825281 conn(0x563257ea4a60 msgr2=0x563257ed80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 -- 172.21.15.87:0/675825281 shutdown_connections 2024-03-21T08:56:59.937 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.934+0000 7f33d3c04740 1 -- 172.21.15.87:0/675825281 wait complete. 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 -- start start 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 Processor -- start 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ef25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ee9660 con 0x563257ee66b0 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257dfde90 con 0x563257ee0b50 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ee97e0 con 0x563257ee1540 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33d3c04740 10 client.0 _ll_register_callbacks cb 0x563257b55ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ef25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ef25a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59030/0 (socket says 172.21.15.87:59030) 2024-03-21T08:56:59.938 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 -- 172.21.15.87:0/852778575 learned_addr learned my addr 172.21.15.87:0/852778575 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33ca59c640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 -- 172.21.15.87:0/852778575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 msgr2=0x563257eeb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 -- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 msgr2=0x563257eeb090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 -- 172.21.15.87:0/852778575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563257e0a9d0 con 0x563257ee66b0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33cad9d640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ef25a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f33c40046e0 tx=0x7f33c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfbddcd314e4e182 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.935+0000 7f33b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33b1ffb640 1 -- 172.21.15.87:0/852778575 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004d20 con 0x563257ee66b0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33b1ffb640 1 -- 172.21.15.87:0/852778575 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f33c40209a0 con 0x563257ee66b0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33c0019b10 con 0x563257ee66b0 2024-03-21T08:56:59.939 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33b1ffb640 1 -- 172.21.15.87:0/852778575 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33c4020ba0 con 0x563257ee66b0 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 10 client.15288 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563257b562d0 con 0x563257ee66b0 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33b1ffb640 1 -- 172.21.15.87:0/852778575 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33c0019b10 con 0x563257ee66b0 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33b1ffb640 1 -- 172.21.15.87:0/852778575 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f33c4027070 con 0x563257ee66b0 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 10 client.15288 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 10 client.15288 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 20 client.15288 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257ef7930 0x563257ef6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 4 client.15288 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 --> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563257e4f160 con 0x563257ef7930 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.936+0000 7f33ca59c640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257ef7930 0x563257ef6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33b1ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:59.940 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33ca59c640 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257ef7930 0x563257ef6940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a2394decb7733f74 server_cookie=a85b9a940a9724aa in_seq=0 out_seq=0 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33b1ffb640 1 -- 172.21.15.87:0/852778575 <== mds.3 v2:172.21.15.100:6834/2548903221 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x563257e4f160 con 0x563257ef7930 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33b1ffb640 10 client.15288 handle_command_reply: tid=0 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 client.15288 shutdown 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 msgr2=0x563257ef25a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ef25a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f33c40046e0 tx=0x7f33c4004a40 comp rx=0 tx=0).stop 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 shutdown_connections 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257ef7930 0x563257ef6940 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee66b0 0x563257ef25a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33b1ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 --2- 172.21.15.87:0/852778575 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.937+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 >> 172.21.15.87:0/852778575 conn(0x563257ea4a60 msgr2=0x563257ed72f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.941 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.938+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 shutdown_connections 2024-03-21T08:56:59.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.938+0000 7f33d3c04740 1 -- 172.21.15.87:0/852778575 wait complete. 2024-03-21T08:56:59.942 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.938+0000 7f33d3c04740 20 client.15288 trim_cache size 0 max 16384 2024-03-21T08:56:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 Processor -- start 2024-03-21T08:56:59.954 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 -- start start 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ef7150 con 0x563257ee0b50 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ef77b0 con 0x563257ee66b0 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33d3c04740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ef7e10 con 0x563257ee1540 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59044/0 (socket says 172.21.15.87:59044) 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 -- 172.21.15.87:0/1411069294 learned_addr learned my addr 172.21.15.87:0/1411069294 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33cad9d640 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 -- 172.21.15.87:0/1411069294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 msgr2=0x563257eeb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.955 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 -- 172.21.15.87:0/1411069294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 msgr2=0x563257ef24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.954+0000 7f33ca59c640 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33ca59c640 1 -- 172.21.15.87:0/1411069294 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563257c206b0 con 0x563257ee0b50 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33cad9d640 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33ca59c640 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eeb090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f33bc02cbc0 tx=0x7f33bc032970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dc10a7b6254257 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33b3fff640 1 -- 172.21.15.87:0/1411069294 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0dd040 con 0x563257ee0b50 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33b3fff640 1 -- 172.21.15.87:0/1411069294 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33bc0bdb00 con 0x563257ee0b50 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33b3fff640 1 -- 172.21.15.87:0/1411069294 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0bb700 con 0x563257ee0b50 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 -- 172.21.15.87:0/1411069294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 msgr2=0x563257eeb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eeb090 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f33bc02cbc0 tx=0x7f33bc032970 comp rx=0 tx=0).stop 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 -- 172.21.15.87:0/1411069294 shutdown_connections 2024-03-21T08:56:59.956 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eeb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 --2- 172.21.15.87:0/1411069294 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eeb090 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 -- 172.21.15.87:0/1411069294 >> 172.21.15.87:0/1411069294 conn(0x563257ea4a60 msgr2=0x563257ee9bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 -- 172.21.15.87:0/1411069294 shutdown_connections 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.955+0000 7f33d3c04740 1 -- 172.21.15.87:0/1411069294 wait complete. 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 -- start start 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 Processor -- start 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eee050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ef8ad0 con 0x563257ee0b50 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ef8470 con 0x563257ee66b0 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x563257ef9130 con 0x563257ee1540 2024-03-21T08:56:59.957 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33d3c04740 10 client.0 _ll_register_callbacks cb 0x7f33c400e450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eee050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33c9d9b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eee050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59054/0 (socket says 172.21.15.87:59054) 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 -- 172.21.15.87:0/2669646039 learned_addr learned my addr 172.21.15.87:0/2669646039 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33cad9d640 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 -- 172.21.15.87:0/2669646039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 msgr2=0x563257eee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 -- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 msgr2=0x563257ef1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 -- 172.21.15.87:0/2669646039 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563257ef1b00 con 0x563257ee0b50 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33ca59c640 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eee050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0eeb40 tx=0x7f33bc0bd580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5554d3f7133903b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:56:59.958 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.956+0000 7f33b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33b1ffb640 1 -- 172.21.15.87:0/2669646039 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0dd030 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33b1ffb640 1 -- 172.21.15.87:0/2669646039 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f33bc0042c0 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f33bc0bdb00 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33b1ffb640 1 -- 172.21.15.87:0/2669646039 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33bc0c1550 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 10 client.15294 fetch_fsmap learned FSMap version 25 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563257f52c40 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33b1ffb640 1 -- 172.21.15.87:0/2669646039 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f33bc0bdb00 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33b1ffb640 1 -- 172.21.15.87:0/2669646039 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f33bc03c030 con 0x563257ee0b50 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 10 client.15294 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 10 client.15294 resolve_mds: resolved name 'e' to daemon mds.e 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 20 client.15294 populate_metadata read hostname 'smithi087' 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257f572a0 0x563257f52e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:56:59.959 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 4 client.15294 mds_command: new command op to 24425 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 --> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563257e494b0 con 0x563257f572a0 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.957+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257f572a0 0x563257f52e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33b1ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33c9d9b640 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257f572a0 0x563257f52e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f5809345428af08d server_cookie=436275185c626ca7 in_seq=0 out_seq=0 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33b1ffb640 1 -- 172.21.15.87:0/2669646039 <== mds.3 v2:172.21.15.100:6834/2548903221 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x563257e494b0 con 0x563257f572a0 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33b1ffb640 10 client.15294 handle_command_reply: tid=0 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 client.15294 shutdown 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 msgr2=0x563257eee050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eee050 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f33bc0eeb40 tx=0x7f33bc0bd580 comp rx=0 tx=0).stop 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 shutdown_connections 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:6834/2548903221,v1:172.21.15.100:6835/2548903221] conn(0x563257f572a0 0x563257f52e60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x563257ee66b0 0x563257ef1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.960 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x563257ee1540 0x563257eee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33b1ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.100:6834/2548903221 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 --2- 172.21.15.87:0/2669646039 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x563257ee0b50 0x563257eee050 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 >> 172.21.15.87:0/2669646039 conn(0x563257ea4a60 msgr2=0x563257eeb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 shutdown_connections 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 1 -- 172.21.15.87:0/2669646039 wait complete. 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.958+0000 7f33d3c04740 20 client.15294 trim_cache size 0 max 16384 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 3, 2024-03-21T08:56:59.961 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24425, 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 109.013698854, 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 139.229073277 2024-03-21T08:56:59.962 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33b40a2f20 msgr2=0x7f33b40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33b40a2f20 0x7f33b40a53e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x563257ee1da0 tx=0x7f33c4004ab0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 msgr2=0x7f33cc197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 0x7f33cc197a00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f33bc02c0b0 tx=0x7f33bc032430 comp rx=0 tx=0).stop 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 shutdown_connections 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f33b40a2f20 0x7f33b40a53e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f33cc102160 0x7f33cc19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f33cc1017a0 0x7f33cc197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 --2- 172.21.15.87:0/1018318735 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f33cc1005b0 0x7f33cc197a00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 >> 172.21.15.87:0/1018318735 conn(0x7f33cc0fbb40 msgr2=0x7f33cc104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 shutdown_connections 2024-03-21T08:56:59.965 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:56:59.959+0000 7f33d0f3b640 1 -- 172.21.15.87:0/1018318735 wait complete. 2024-03-21T08:56:59.966 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.f status 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.073+0000 7f8c22b59640 1 -- 172.21.15.87:0/2889070912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c102160 msgr2=0x7f8c1c1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.073+0000 7f8c22b59640 1 --2- 172.21.15.87:0/2889070912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c102160 0x7f8c1c1089f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8c04009900 tx=0x7f8c0402bc30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- 172.21.15.87:0/2889070912 shutdown_connections 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 --2- 172.21.15.87:0/2889070912 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c102160 0x7f8c1c1089f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 --2- 172.21.15.87:0/2889070912 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 --2- 172.21.15.87:0/2889070912 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c1c1005b0 0x7f8c1c1009d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- 172.21.15.87:0/2889070912 >> 172.21.15.87:0/2889070912 conn(0x7f8c1c0fbb40 msgr2=0x7f8c1c0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- 172.21.15.87:0/2889070912 shutdown_connections 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- 172.21.15.87:0/2889070912 wait complete. 2024-03-21T08:57:00.071 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 Processor -- start 2024-03-21T08:57:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- start start 2024-03-21T08:57:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c1c1005b0 0x7f8c1c196850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c196d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c102160 0x7f8c1c19de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10c210 con 0x7f8c1c1017a0 2024-03-21T08:57:00.072 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10c090 con 0x7f8c1c1005b0 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c22b59640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c10c390 con 0x7f8c1c102160 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c196d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c196d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59060/0 (socket says 172.21.15.87:59060) 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c1bfff640 1 -- 172.21.15.87:0/512807550 learned_addr learned my addr 172.21.15.87:0/512807550 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c210f6640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c102160 0x7f8c1c19de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.074+0000 7f8c208f5640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c1c1005b0 0x7f8c1c196850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c1bfff640 1 -- 172.21.15.87:0/512807550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c102160 msgr2=0x7f8c1c19de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c102160 0x7f8c1c19de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c1bfff640 1 -- 172.21.15.87:0/512807550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c1c1005b0 msgr2=0x7f8c1c196850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c1c1005b0 0x7f8c1c196850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.073 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c1bfff640 1 -- 172.21.15.87:0/512807550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c19e470 con 0x7f8c1c1017a0 2024-03-21T08:57:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c196d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00ea60 tx=0x7f8c0c00ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c19ffb640 1 -- 172.21.15.87:0/512807550 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c00cde0 con 0x7f8c1c1017a0 2024-03-21T08:57:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c19ffb640 1 -- 172.21.15.87:0/512807550 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c0035f0 con 0x7f8c1c1017a0 2024-03-21T08:57:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c19e760 con 0x7f8c1c1017a0 2024-03-21T08:57:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c19ffb640 1 -- 172.21.15.87:0/512807550 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c010670 con 0x7f8c1c1017a0 2024-03-21T08:57:00.074 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.075+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c19ecb0 con 0x7f8c1c1017a0 2024-03-21T08:57:00.082 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.077+0000 7f8c19ffb640 1 -- 172.21.15.87:0/512807550 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f8c0c0037a0 con 0x7f8c1c1017a0 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.077+0000 7f8c25811740 1 Processor -- start 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 -- start start 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c664cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c6651c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c6654730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c65b1c20 con 0x55d1c66522a0 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c655aa40 con 0x55d1c664c740 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c25811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c65adcc0 con 0x55d1c664d130 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c6654730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c210f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c664cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c6651c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.083 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c210f6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c664cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:40528/0 (socket says 172.21.15.87:40528) 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c6654730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59072/0 (socket says 172.21.15.87:59072) 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c210f6640 1 -- 172.21.15.87:0/3561675561 learned_addr learned my addr 172.21.15.87:0/3561675561 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c19ffb640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8c080a2f20 0x7f8c080a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c210f6640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8c080a2f20 0x7f8c080a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 -- 172.21.15.87:0/3561675561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 msgr2=0x55d1c664cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c664cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 -- 172.21.15.87:0/3561675561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 msgr2=0x55d1c6654730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c6654730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 -- 172.21.15.87:0/3561675561 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c657cf90 con 0x55d1c664d130 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c19ffb640 1 -- 172.21.15.87:0/512807550 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c0c0cf6a0 con 0x7f8c1c1017a0 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.078+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c6651c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c14005fd0 tx=0x7f8c140118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af8e96e7b672d6af server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.084 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c210f6640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8c080a2f20 0x7f8c080a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005c40 tx=0x7f8c0402bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c12ffd640 1 -- 172.21.15.87:0/3561675561 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c14004d20 con 0x55d1c664d130 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c12ffd640 1 -- 172.21.15.87:0/3561675561 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c14003e50 con 0x55d1c664d130 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c12ffd640 1 -- 172.21.15.87:0/3561675561 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1400f7a0 con 0x55d1c664d130 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 -- 172.21.15.87:0/3561675561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 msgr2=0x55d1c6651c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c6651c30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8c14005fd0 tx=0x7f8c140118d0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 -- 172.21.15.87:0/3561675561 shutdown_connections 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c6654730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c6651c30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 --2- 172.21.15.87:0/3561675561 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c664cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 -- 172.21.15.87:0/3561675561 >> 172.21.15.87:0/3561675561 conn(0x55d1c6614a60 msgr2=0x55d1c66480b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 -- 172.21.15.87:0/3561675561 shutdown_connections 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.079+0000 7f8c25811740 1 -- 172.21.15.87:0/3561675561 wait complete. 2024-03-21T08:57:00.085 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 -- start start 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 Processor -- start 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6656c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c66571a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c665e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6655250 con 0x55d1c66522a0 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c656de90 con 0x55d1c664c740 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c66553d0 con 0x55d1c664d130 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c25811740 10 client.0 _ll_register_callbacks cb 0x55d1c62c5ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c665e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c665e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59076/0 (socket says 172.21.15.87:59076) 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c208f5640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6656c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 -- 172.21.15.87:0/3294977509 learned_addr learned my addr 172.21.15.87:0/3294977509 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.086 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c66571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 -- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 msgr2=0x55d1c66571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c66571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 -- 172.21.15.87:0/3294977509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 msgr2=0x55d1c6656c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6656c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c210f6640 1 -- 172.21.15.87:0/3294977509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1c657a9d0 con 0x55d1c66522a0 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.080+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6656c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c210f6640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c665e1e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8c04036040 tx=0x7f8c04009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73927990dc8e15b5 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3294977509 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c04052060 con 0x55d1c66522a0 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3294977509 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c04005300 con 0x55d1c66522a0 2024-03-21T08:57:00.087 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c14003e50 con 0x55d1c66522a0 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3294977509 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c04005500 con 0x55d1c66522a0 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 10 client.15306 fetch_fsmap learned FSMap version 25 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d1c62c62d0 con 0x55d1c66522a0 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3294977509 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c14003e50 con 0x55d1c66522a0 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3294977509 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8c040424d0 con 0x55d1c66522a0 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 10 client.15306 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 10 client.15306 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 20 client.15306 populate_metadata read hostname 'smithi087' 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c6662520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 4 client.15306 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 --> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d1c65bf160 con 0x55d1c6663490 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.081+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c6662520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.088 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.082+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c6662520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=85e00abbd8436190 server_cookie=f4ad568eed69ff73 in_seq=0 out_seq=0 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.082+0000 7f8bfbfff640 10 client.15306 ms_handle_connect on v2:172.21.15.105:6834/1031026542 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.082+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3294977509 <== mds.2 v2:172.21.15.105:6834/1031026542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55d1c65bf160 con 0x55d1c6663490 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.082+0000 7f8bfbfff640 10 client.15306 handle_command_reply: tid=0 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.082+0000 7f8c25811740 1 client.15306 shutdown 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 msgr2=0x55d1c665e1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c665e1e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f8c04036040 tx=0x7f8c04009260 comp rx=0 tx=0).stop 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 shutdown_connections 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c6662520 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c66522a0 0x55d1c665e1e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8bfbfff640 0 client.15306 ms_handle_reset on v2:172.21.15.105:6834/1031026542 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c664d130 0x55d1c66571a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.089 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 --2- 172.21.15.87:0/3294977509 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6656c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 >> 172.21.15.87:0/3294977509 conn(0x55d1c6614a60 msgr2=0x55d1c66472f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 shutdown_connections 2024-03-21T08:57:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 1 -- 172.21.15.87:0/3294977509 wait complete. 2024-03-21T08:57:00.090 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.083+0000 7f8c25811740 20 client.15306 trim_cache size 0 max 16384 2024-03-21T08:57:00.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.099+0000 7f8c25811740 1 Processor -- start 2024-03-21T08:57:00.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.099+0000 7f8c25811740 1 -- start start 2024-03-21T08:57:00.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c25811740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6657100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c25811740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c25811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c6654690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.100 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c25811740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6662cb0 con 0x55d1c664d130 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c25811740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6663310 con 0x55d1c664c740 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c25811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6663970 con 0x55d1c66522a0 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59088/0 (socket says 172.21.15.87:59088) 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 -- 172.21.15.87:0/576578791 learned_addr learned my addr 172.21.15.87:0/576578791 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c210f6640 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c6654690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c208f5640 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6657100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 -- 172.21.15.87:0/576578791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 msgr2=0x55d1c6654690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c6654690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 -- 172.21.15.87:0/576578791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 msgr2=0x55d1c6657100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6657100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.101 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 -- 172.21.15.87:0/576578791 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d1c63906b0 con 0x55d1c664d130 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665e1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c01f000 tx=0x7f8c0c09bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76d5a5b3adc5e6a server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c127fc640 1 -- 172.21.15.87:0/576578791 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c09c500 con 0x55d1c664d130 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c127fc640 1 -- 172.21.15.87:0/576578791 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c09be00 con 0x55d1c664d130 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.100+0000 7f8c127fc640 1 -- 172.21.15.87:0/576578791 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0bab70 con 0x55d1c664d130 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 -- 172.21.15.87:0/576578791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 msgr2=0x55d1c665e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665e1a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c01f000 tx=0x7f8c0c09bc80 comp rx=0 tx=0).stop 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 -- 172.21.15.87:0/576578791 shutdown_connections 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c6654690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665e1a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 --2- 172.21.15.87:0/576578791 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c6657100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 -- 172.21.15.87:0/576578791 >> 172.21.15.87:0/576578791 conn(0x55d1c6614a60 msgr2=0x55d1c6655800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.102 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 -- 172.21.15.87:0/576578791 shutdown_connections 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 -- 172.21.15.87:0/576578791 wait complete. 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 -- start start 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.101+0000 7f8c25811740 1 Processor -- start 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c665a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c665d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6664630 con 0x55d1c664d130 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6663fd0 con 0x55d1c664c740 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1c6664c90 con 0x55d1c66522a0 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 10 client.0 _ll_register_callbacks cb 0x7f8c04009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59100/0 (socket says 172.21.15.87:59100) 2024-03-21T08:57:00.103 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 -- 172.21.15.87:0/3410900906 learned_addr learned my addr 172.21.15.87:0/3410900906 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c665a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c210f6640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c665d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 -- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 msgr2=0x55d1c665d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c665d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 -- 172.21.15.87:0/3410900906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 msgr2=0x55d1c665a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c665a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 -- 172.21.15.87:0/3410900906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d1c665dd80 con 0x55d1c664d130 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c1bfff640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665a7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0cd540 tx=0x7f8c0c003b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7d33d7a5b8890b9 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8bfbfff640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3410900906 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c09ff50 con 0x55d1c664d130 2024-03-21T08:57:00.104 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3410900906 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f8c0c098470 con 0x55d1c664d130 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0c09be00 con 0x55d1c664d130 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3410900906 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0c0a4810 con 0x55d1c664d130 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 10 client.15312 fetch_fsmap learned FSMap version 25 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.102+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55d1c66bee30 con 0x55d1c664d130 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3410900906 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f8c0c09be00 con 0x55d1c664d130 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3410900906 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7f8c0c0cfb50 con 0x55d1c664d130 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 10 client.15312 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 10 client.15312 resolve_mds: resolved name 'f' to daemon mds.f 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 20 client.15312 populate_metadata read hostname 'smithi087' 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c66bf000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 4 client.15312 mds_command: new command op to 24451 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 --> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d1c65b94b0 con 0x55d1c6663490 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c66bf000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.105 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8bfbfff640 10 client.15312 ms_handle_connect on v2:172.21.15.105:6834/1031026542 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c208f5640 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c66bf000 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2292f991346d7c2 server_cookie=aa22c50578e17fcc in_seq=0 out_seq=0 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8bfbfff640 1 -- 172.21.15.87:0/3410900906 <== mds.2 v2:172.21.15.105:6834/1031026542 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d1c65b94b0 con 0x55d1c6663490 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8bfbfff640 10 client.15312 handle_command_reply: tid=0 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.103+0000 7f8c25811740 1 client.15312 shutdown 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 msgr2=0x55d1c665a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665a7e0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c0cd540 tx=0x7f8c0c003b30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 shutdown_connections 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55d1c66522a0 0x55d1c665d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.105:6834/1031026542,v1:172.21.15.105:6835/1031026542] conn(0x55d1c6663490 0x55d1c66bf000 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55d1c664d130 0x55d1c665a7e0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8bfbfff640 0 client.15312 ms_handle_reset on v2:172.21.15.105:6834/1031026542 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 --2- 172.21.15.87:0/3410900906 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55d1c664c740 0x55d1c665a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.106 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 >> 172.21.15.87:0/3410900906 conn(0x55d1c6614a60 msgr2=0x55d1c6662520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 shutdown_connections 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 1 -- 172.21.15.87:0/3410900906 wait complete. 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c25811740 20 client.15312 trim_cache size 0 max 16384 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 2, 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24451, 2024-03-21T08:57:00.107 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 110.166758571, 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 136.54040859200001 2024-03-21T08:57:00.108 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8c080a2f20 msgr2=0x7f8c080a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8c080a2f20 0x7f8c080a53e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c04005c40 tx=0x7f8c0402bc30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 msgr2=0x7f8c1c196d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c196d90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c00ea60 tx=0x7f8c0c00ef20 comp rx=0 tx=0).stop 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 shutdown_connections 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f8c080a2f20 0x7f8c080a53e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.109 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f8c1c102160 0x7f8c1c19de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f8c1c1017a0 0x7f8c1c196d90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 --2- 172.21.15.87:0/512807550 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f8c1c1005b0 0x7f8c1c196850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 >> 172.21.15.87:0/512807550 conn(0x7f8c1c0fbb40 msgr2=0x7f8c1c104aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 shutdown_connections 2024-03-21T08:57:00.110 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.104+0000 7f8c22b59640 1 -- 172.21.15.87:0/512807550 wait complete. 2024-03-21T08:57:00.111 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-03-21T08:57:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 -- 172.21.15.87:0/2638432253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac103280 msgr2=0x7fadac109b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.215 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2638432253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac103280 0x7fadac109b10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fada4009900 tx=0x7fada402bc30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 -- 172.21.15.87:0/2638432253 shutdown_connections 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2638432253 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac103280 0x7fadac109b10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2638432253 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac1028c0 0x7fadac102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2638432253 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadac1016d0 0x7fadac101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 -- 172.21.15.87:0/2638432253 >> 172.21.15.87:0/2638432253 conn(0x7fadac0fcc40 msgr2=0x7fadac0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 -- 172.21.15.87:0/2638432253 shutdown_connections 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.218+0000 7fadb18c6640 1 -- 172.21.15.87:0/2638432253 wait complete. 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 Processor -- start 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 -- start start 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac1016d0 0x7fadac197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.216 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 0x7fadac197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadac103280 0x7fadac19ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10d330 con 0x7fadac1028c0 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10d1b0 con 0x7fadac1016d0 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadb18c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac10d4b0 con 0x7fadac103280 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 0x7fadac197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 0x7fadac197e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59122/0 (socket says 172.21.15.87:59122) 2024-03-21T08:57:00.217 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 -- 172.21.15.87:0/2000754283 learned_addr learned my addr 172.21.15.87:0/2000754283 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaaffd640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac1016d0 0x7fadac197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadab7fe640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadac103280 0x7fadac19ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 -- 172.21.15.87:0/2000754283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadac103280 msgr2=0x7fadac19ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadac103280 0x7fadac19ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 -- 172.21.15.87:0/2000754283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac1016d0 msgr2=0x7fadac197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac1016d0 0x7fadac197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.219+0000 7fadaa7fc640 1 -- 172.21.15.87:0/2000754283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac19f570 con 0x7fadac1028c0 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.220+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 0x7fadac197e90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fada000e9f0 tx=0x7fada000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.218 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.220+0000 7fadb08c4640 1 -- 172.21.15.87:0/2000754283 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada000cd80 con 0x7fadac1028c0 2024-03-21T08:57:00.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.220+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac19f860 con 0x7fadac1028c0 2024-03-21T08:57:00.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.220+0000 7fadb08c4640 1 -- 172.21.15.87:0/2000754283 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fada0010910 con 0x7fadac1028c0 2024-03-21T08:57:00.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.220+0000 7fadb08c4640 1 -- 172.21.15.87:0/2000754283 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00188f0 con 0x7fadac1028c0 2024-03-21T08:57:00.219 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.220+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac19fdb0 con 0x7fadac1028c0 2024-03-21T08:57:00.226 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb08c4640 1 -- 172.21.15.87:0/2000754283 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fada0010ac0 con 0x7fadac1028c0 2024-03-21T08:57:00.227 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 Processor -- start 2024-03-21T08:57:00.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 -- start start 2024-03-21T08:57:00.229 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391c740 0x55be8391cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be83921c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 0x55be83924730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83881c20 con 0x55be839222a0 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8382aa40 con 0x55be8391d130 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.222+0000 7fadb4591740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8387dcc0 con 0x55be8391c740 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 0x55be83924730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadab7fe640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391c740 0x55be8391cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.230 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be83921c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 0x55be83924730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59138/0 (socket says 172.21.15.87:59138) 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 -- 172.21.15.87:0/395818467 learned_addr learned my addr 172.21.15.87:0/395818467 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadb08c4640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad940a2f20 0x7fad940a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadab7fe640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad940a2f20 0x7fad940a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 -- 172.21.15.87:0/395818467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391c740 msgr2=0x55be8391cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391c740 0x55be8391cb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 -- 172.21.15.87:0/395818467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 msgr2=0x55be83921c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be83921c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.231 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadb08c4640 1 -- 172.21.15.87:0/2000754283 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fada00cf7a0 con 0x7fadac1028c0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 -- 172.21.15.87:0/395818467 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be8384cf90 con 0x55be839222a0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaaffd640 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be83921c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadab7fe640 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391c740 0x55be8391cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 0x55be83924730 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fada00cd210 tx=0x7fada001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ce871e04d78324b server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fadab7fe640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad940a2f20 0x7fad940a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fada4005ab0 tx=0x7fada402bc30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fad91ffb640 1 -- 172.21.15.87:0/395818467 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada00a4910 con 0x55be839222a0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fad91ffb640 1 -- 172.21.15.87:0/395818467 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fada005fde0 con 0x55be839222a0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.223+0000 7fad91ffb640 1 -- 172.21.15.87:0/395818467 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada009b400 con 0x55be839222a0 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 -- 172.21.15.87:0/395818467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 msgr2=0x55be83924730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 0x55be83924730 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fada00cd210 tx=0x7fada001f000 comp rx=0 tx=0).stop 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 -- 172.21.15.87:0/395818467 shutdown_connections 2024-03-21T08:57:00.232 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be839222a0 0x55be83924730 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be83921c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 --2- 172.21.15.87:0/395818467 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391c740 0x55be8391cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 -- 172.21.15.87:0/395818467 >> 172.21.15.87:0/395818467 conn(0x55be838e4a60 msgr2=0x55be839180b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 -- 172.21.15.87:0/395818467 shutdown_connections 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 -- 172.21.15.87:0/395818467 wait complete. 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 -- start start 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.224+0000 7fadb4591740 1 Processor -- start 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83926b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be839270d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be8392e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83925190 con 0x55be8391c740 2024-03-21T08:57:00.233 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be8383de90 con 0x55be8391d130 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83925310 con 0x55be839222a0 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 10 client.0 _ll_register_callbacks cb 0x55be83595ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83926b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83926b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59152/0 (socket says 172.21.15.87:59152) 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 -- 172.21.15.87:0/1831025082 learned_addr learned my addr 172.21.15.87:0/1831025082 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadab7fe640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be8392e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be839270d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 -- 172.21.15.87:0/1831025082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 msgr2=0x55be8392e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be8392e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.234 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 -- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 msgr2=0x55be839270d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be839270d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 -- 172.21.15.87:0/1831025082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be8384a9d0 con 0x55be8391c740 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadaaffd640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83926b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011ce0 tx=0x7fad9c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=951d89aa37fe6f49 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fad8affd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fad8affd640 1 -- 172.21.15.87:0/1831025082 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c009e50 con 0x55be8391c740 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fad8affd640 1 -- 172.21.15.87:0/1831025082 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fad9c003970 con 0x55be8391c740 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fada005fde0 con 0x55be8391c740 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fad8affd640 1 -- 172.21.15.87:0/1831025082 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fad9c020030 con 0x55be8391c740 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 10 client.15324 fetch_fsmap learned FSMap version 25 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.225+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be835962d0 con 0x55be8391c740 2024-03-21T08:57:00.235 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fad8affd640 1 -- 172.21.15.87:0/1831025082 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fada005fde0 con 0x55be8391c740 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fad8affd640 1 -- 172.21.15.87:0/1831025082 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fad9c0045d0 con 0x55be8391c740 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadb4591740 10 client.15324 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadb4591740 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadb4591740 20 client.15324 populate_metadata read hostname 'smithi087' 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadb4591740 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be839333f0 0x55be83932450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadb4591740 4 client.15324 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 --> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be8388f160 con 0x55be839333f0 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be839333f0 0x55be83932450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fad8affd640 10 client.15324 ms_handle_connect on v2:172.21.15.100:6838/214101858 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.226+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be839333f0 0x55be83932450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9050e0d79a71a45e server_cookie=3d991e9728eef99 in_seq=0 out_seq=0 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fad8affd640 1 -- 172.21.15.87:0/1831025082 <== mds.1 v2:172.21.15.100:6838/214101858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x55be8388f160 con 0x55be839333f0 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fad8affd640 10 client.15324 handle_command_reply: tid=0 2024-03-21T08:57:00.236 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 client.15324 shutdown 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 msgr2=0x55be83926b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83926b90 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fad9c011ce0 tx=0x7fad9c00f790 comp rx=0 tx=0).stop 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 shutdown_connections 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be8392e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391d130 0x55be839270d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be839333f0 0x55be83932450 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 --2- 172.21.15.87:0/1831025082 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83926b90 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fad8affd640 0 client.15324 ms_handle_reset on v2:172.21.15.100:6838/214101858 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 >> 172.21.15.87:0/1831025082 conn(0x55be838e4a60 msgr2=0x55be839172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 shutdown_connections 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 1 -- 172.21.15.87:0/1831025082 wait complete. 2024-03-21T08:57:00.237 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.227+0000 7fadb4591740 20 client.15324 trim_cache size 0 max 16384 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 Processor -- start 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 -- start start 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83932450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391d130 0x55be8393b890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be839222a0 0x55be8393bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83932c10 con 0x55be8391c740 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83933270 con 0x55be839222a0 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadb4591740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be839338d0 con 0x55be8391d130 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83932450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.245 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83932450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59166/0 (socket says 172.21.15.87:59166) 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 -- 172.21.15.87:0/3951713415 learned_addr learned my addr 172.21.15.87:0/3951713415 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391d130 0x55be8393b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadab7fe640 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be839222a0 0x55be8393bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 -- 172.21.15.87:0/3951713415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391d130 msgr2=0x55be8393b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391d130 0x55be8393b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 -- 172.21.15.87:0/3951713415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be839222a0 msgr2=0x55be8393bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be839222a0 0x55be8393bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 -- 172.21.15.87:0/3951713415 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be836606b0 con 0x55be8391c740 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391d130 0x55be8393b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fadaaffd640 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83932450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00ff90 tx=0x7fad9c009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8cf74a434ab01e0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.244+0000 7fad917fa640 1 -- 172.21.15.87:0/3951713415 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c003970 con 0x55be8391c740 2024-03-21T08:57:00.246 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fad917fa640 1 -- 172.21.15.87:0/3951713415 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad9c0256f0 con 0x55be8391c740 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fad917fa640 1 -- 172.21.15.87:0/3951713415 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad9c005180 con 0x55be8391c740 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 -- 172.21.15.87:0/3951713415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 msgr2=0x55be83932450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83932450 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fad9c00ff90 tx=0x7fad9c009570 comp rx=0 tx=0).stop 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 -- 172.21.15.87:0/3951713415 shutdown_connections 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be839222a0 0x55be8393bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be8391d130 0x55be8393b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 --2- 172.21.15.87:0/3951713415 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391c740 0x55be83932450 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 -- 172.21.15.87:0/3951713415 >> 172.21.15.87:0/3951713415 conn(0x55be838e4a60 msgr2=0x55be839172f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 -- 172.21.15.87:0/3951713415 shutdown_connections 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 -- 172.21.15.87:0/3951713415 wait complete. 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.245+0000 7fadb4591740 1 -- start start 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 Processor -- start 2024-03-21T08:57:00.247 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391c740 0x55be8398af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 0x55be8398b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be839924b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83934590 con 0x55be8391d130 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83933f30 con 0x55be8391c740 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be83934bf0 con 0x55be839222a0 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 10 client.0 _ll_register_callbacks cb 0x7fad9c009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 0x55be8398b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 0x55be8398b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59168/0 (socket says 172.21.15.87:59168) 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 -- 172.21.15.87:0/254836460 learned_addr learned my addr 172.21.15.87:0/254836460 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaaffd640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391c740 0x55be8398af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadab7fe640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be839924b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 -- 172.21.15.87:0/254836460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 msgr2=0x55be839924b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.248 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be839924b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 -- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391c740 msgr2=0x55be8398af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391c740 0x55be8398af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 -- 172.21.15.87:0/254836460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be83992aa0 con 0x55be8391d130 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadaa7fc640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 0x55be8398b470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fada00cd640 tx=0x7fada009bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76594b87e7a3024 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fad8affd640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fad8affd640 1 -- 172.21.15.87:0/254836460 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0098620 con 0x55be8391d130 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fad8affd640 1 -- 172.21.15.87:0/254836460 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fada00610a0 con 0x55be8391d130 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.246+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fad9c0256f0 con 0x55be8391d130 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fad8affd640 1 -- 172.21.15.87:0/254836460 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fada00612a0 con 0x55be8391d130 2024-03-21T08:57:00.249 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 10 client.15330 fetch_fsmap learned FSMap version 25 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55be83993da0 con 0x55be8391d130 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fad8affd640 1 -- 172.21.15.87:0/254836460 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fad9c0256f0 con 0x55be8391d130 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fad8affd640 1 -- 172.21.15.87:0/254836460 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7fada0010430 con 0x55be8391d130 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 10 client.15330 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 10 client.15330 resolve_mds: resolved name 'b' to daemon mds.b 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 20 client.15330 populate_metadata read hostname 'smithi087' 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be83997f40 0x55be83993f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 4 client.15330 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 --> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be839254a0 con 0x55be83997f40 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadaaffd640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be83997f40 0x55be83993f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fad8affd640 10 client.15330 ms_handle_connect on v2:172.21.15.100:6838/214101858 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.247+0000 7fadaaffd640 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be83997f40 0x55be83993f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=326f65e3155753a6 server_cookie=2a49b0a7eeb74e6 in_seq=0 out_seq=0 2024-03-21T08:57:00.250 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fad8affd640 1 -- 172.21.15.87:0/254836460 <== mds.1 v2:172.21.15.100:6838/214101858 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55be839254a0 con 0x55be83997f40 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fad8affd640 10 client.15330 handle_command_reply: tid=0 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 client.15330 shutdown 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 msgr2=0x55be8398b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 0x55be8398b470 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fada00cd640 tx=0x7fada009bfb0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 shutdown_connections 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x55be839222a0 0x55be839924b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:6838/214101858,v1:172.21.15.100:6839/214101858] conn(0x55be83997f40 0x55be83993f70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x55be8391d130 0x55be8398b470 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 --2- 172.21.15.87:0/254836460 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x55be8391c740 0x55be8398af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fad8affd640 0 client.15330 ms_handle_reset on v2:172.21.15.100:6838/214101858 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 >> 172.21.15.87:0/254836460 conn(0x55be838e4a60 msgr2=0x55be8391b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 shutdown_connections 2024-03-21T08:57:00.251 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 1 -- 172.21.15.87:0/254836460 wait complete. 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb4591740 20 client.15330 trim_cache size 0 max 16384 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 1, 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24482, 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 110.703108282, 2024-03-21T08:57:00.252 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:57:00.253 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:57:00.253 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:57:00.253 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 119.87826866100001 2024-03-21T08:57:00.253 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad940a2f20 msgr2=0x7fad940a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad940a2f20 0x7fad940a53e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fada4005ab0 tx=0x7fada402bc30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 msgr2=0x7fadac197e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 0x7fadac197e90 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fada000e9f0 tx=0x7fada000eeb0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 shutdown_connections 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fad940a2f20 0x7fad940a53e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fadac103280 0x7fadac19ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fadac1028c0 0x7fadac197e90 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 --2- 172.21.15.87:0/2000754283 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fadac1016d0 0x7fadac197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.254 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.248+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 >> 172.21.15.87:0/2000754283 conn(0x7fadac0fcc40 msgr2=0x7fadac105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.249+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 shutdown_connections 2024-03-21T08:57:00.255 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.249+0000 7fadb18c6640 1 -- 172.21.15.87:0/2000754283 wait complete. 2024-03-21T08:57:00.255 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:00 smithi087 ceph-mon[32385]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:00.255 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:00 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2415551941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:57:00.255 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:00 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1381819186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:57:00.255 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:00 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2019517876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:57:00.256 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-03-21T08:57:00.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:00 smithi100 ceph-mon[36306]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:00.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:00 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2415551941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:57:00.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:00 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1381819186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:57:00.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:00 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2019517876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 -- 172.21.15.87:0/2990680716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 msgr2=0x7ff7401009e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 --2- 172.21.15.87:0/2990680716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff7401009e0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff730009900 tx=0x7ff73002bc30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 -- 172.21.15.87:0/2990680716 shutdown_connections 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 --2- 172.21.15.87:0/2990680716 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740102170 0x7ff740108a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 --2- 172.21.15.87:0/2990680716 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7401017b0 0x7ff740101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 --2- 172.21.15.87:0/2990680716 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff7401009e0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 -- 172.21.15.87:0/2990680716 >> 172.21.15.87:0/2990680716 conn(0x7ff7400fbb50 msgr2=0x7ff7400fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.361 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 -- 172.21.15.87:0/2990680716 shutdown_connections 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 -- 172.21.15.87:0/2990680716 wait complete. 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 Processor -- start 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.364+0000 7ff7452c6640 1 -- start start 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff74006a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7401017b0 0x7ff74019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740102170 0x7ff74019ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74010c230 con 0x7ff7401005c0 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74010c0b0 con 0x7ff740102170 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74010c3b0 con 0x7ff7401017b0 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff74006a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff74006a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59184/0 (socket says 172.21.15.87:59184) 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 -- 172.21.15.87:0/1993699771 learned_addr learned my addr 172.21.15.87:0/1993699771 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.362 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff737fff640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7401017b0 0x7ff74019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740102170 0x7ff74019ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 -- 172.21.15.87:0/1993699771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7401017b0 msgr2=0x7ff74019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7401017b0 0x7ff74019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 -- 172.21.15.87:0/1993699771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740102170 msgr2=0x7ff74019ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740102170 0x7ff74019ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 -- 172.21.15.87:0/1993699771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74019f660 con 0x7ff7401005c0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73effd640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff74006a7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff730000c00 tx=0x7ff73002c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73cff9640 1 -- 172.21.15.87:0/1993699771 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730041070 con 0x7ff7401005c0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74019f890 con 0x7ff7401005c0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73cff9640 1 -- 172.21.15.87:0/1993699771 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff730004450 con 0x7ff7401005c0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff73cff9640 1 -- 172.21.15.87:0/1993699771 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730034440 con 0x7ff7401005c0 2024-03-21T08:57:00.363 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.365+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74019fd80 con 0x7ff7401005c0 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.368+0000 7ff73cff9640 1 -- 172.21.15.87:0/1993699771 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff730044070 con 0x7ff7401005c0 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.368+0000 7ff747f92740 1 Processor -- start 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.368+0000 7ff747f92740 1 -- start start 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff747f92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f265fb50 0x5582f265ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff747f92740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 0x5582f2665040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff747f92740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f2667b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff747f92740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f25c0c20 con 0x5582f2660540 2024-03-21T08:57:00.370 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff747f92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2569a40 con 0x5582f265fb50 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff747f92740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f25bccc0 con 0x5582f26656b0 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 0x5582f2665040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f265fb50 0x5582f265ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 0x5582f2665040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59194/0 (socket says 172.21.15.87:59194) 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 -- 172.21.15.87:0/4200114065 learned_addr learned my addr 172.21.15.87:0/4200114065 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff737fff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f2667b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73cff9640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff72c0a2f20 0x7ff72c0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73cff9640 1 -- 172.21.15.87:0/1993699771 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff7300f0e90 con 0x7ff7401005c0 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 -- 172.21.15.87:0/4200114065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 msgr2=0x5582f2667b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f2667b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 -- 172.21.15.87:0/4200114065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f265fb50 msgr2=0x5582f265ff70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.371 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f265fb50 0x5582f265ff70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 -- 172.21.15.87:0/4200114065 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582f258bf90 con 0x5582f2660540 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff737fff640 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f2667b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f265fb50 0x5582f265ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff72c0a2f20 0x7ff72c0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff73effd640 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 0x5582f2665040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7300bd430 tx=0x7ff73002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfce3b9afa205dbd server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff7357fa640 1 -- 172.21.15.87:0/4200114065 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73004b070 con 0x5582f2660540 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff7357fa640 1 -- 172.21.15.87:0/4200114065 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff730004c80 con 0x5582f2660540 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.369+0000 7ff7357fa640 1 -- 172.21.15.87:0/4200114065 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300c6000 con 0x5582f2660540 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 -- 172.21.15.87:0/4200114065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 msgr2=0x5582f2665040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff72c0a2f20 0x7ff72c0a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5582f2660da0 tx=0x7ff738014040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 0x5582f2665040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff7300bd430 tx=0x7ff73002bc30 comp rx=0 tx=0).stop 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 -- 172.21.15.87:0/4200114065 shutdown_connections 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f2667b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f2660540 0x5582f2665040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 --2- 172.21.15.87:0/4200114065 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f265fb50 0x5582f265ff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 -- 172.21.15.87:0/4200114065 >> 172.21.15.87:0/4200114065 conn(0x5582f2623a60 msgr2=0x5582f26570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 -- 172.21.15.87:0/4200114065 shutdown_connections 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 -- 172.21.15.87:0/4200114065 wait complete. 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.370+0000 7ff747f92740 1 -- start start 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 Processor -- start 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2669cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f2660540 0x5582f266a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f26656b0 0x5582f2671230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f25c0c20 con 0x5582f265fb50 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f257ce90 con 0x5582f26656b0 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2668420 con 0x5582f2660540 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 10 client.0 _ll_register_callbacks cb 0x5582f22d4ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:57:00.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2669cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2669cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59206/0 (socket says 172.21.15.87:59206) 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 -- 172.21.15.87:0/61326096 learned_addr learned my addr 172.21.15.87:0/61326096 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff737fff640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f2660540 0x5582f266a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f26656b0 0x5582f2671230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 -- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f2660540 msgr2=0x5582f266a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f2660540 0x5582f266a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 -- 172.21.15.87:0/61326096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f26656b0 msgr2=0x5582f2671230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f26656b0 0x5582f2671230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 -- 172.21.15.87:0/61326096 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582f25899d0 con 0x5582f265fb50 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff73effd640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2669cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff7300f0040 tx=0x7ff7300bd7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab9d5cbfa1f58b14 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff7267fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:57:00.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff7267fc640 1 -- 172.21.15.87:0/61326096 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73004b040 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff7267fc640 1 -- 172.21.15.87:0/61326096 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff7300035f0 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.371+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff730004c80 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff7267fc640 1 -- 172.21.15.87:0/61326096 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7300bd270 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 10 client.15342 fetch_fsmap learned FSMap version 25 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5582f22d52d0 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff7267fc640 1 -- 172.21.15.87:0/61326096 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff730004c80 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff7267fc640 1 -- 172.21.15.87:0/61326096 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff7300f4120 con 0x5582f265fb50 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 10 client.15342 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 10 client.15342 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 20 client.15342 populate_metadata read hostname 'smithi087' 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f2676270 0x5582f2675390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 4 client.15342 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-03-21T08:57:00.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 --> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582f25ce160 con 0x5582f2676270 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff737fff640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f2676270 0x5582f2675390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff7267fc640 10 client.15342 ms_handle_connect on v2:172.21.15.105:6838/892092628 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.372+0000 7ff737fff640 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f2676270 0x5582f2675390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80e5e324e5b1a5cf server_cookie=8406bc409c49bf1d in_seq=0 out_seq=0 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff7267fc640 1 -- 172.21.15.87:0/61326096 <== mds.0 v2:172.21.15.105:6838/892092628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+21660 (crc 0 0 0) 0x5582f25ce160 con 0x5582f2676270 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff7267fc640 10 client.15342 handle_command_reply: tid=0 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 client.15342 shutdown 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 msgr2=0x5582f2669cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2669cb0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff7300f0040 tx=0x7ff7300bd7a0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 shutdown_connections 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f2676270 0x5582f2675390 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f26656b0 0x5582f2671230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff7267fc640 0 client.15342 ms_handle_reset on v2:172.21.15.105:6838/892092628 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f2660540 0x5582f266a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 --2- 172.21.15.87:0/61326096 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2669cb0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 >> 172.21.15.87:0/61326096 conn(0x5582f2623a60 msgr2=0x5582f26562f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 shutdown_connections 2024-03-21T08:57:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 1 -- 172.21.15.87:0/61326096 wait complete. 2024-03-21T08:57:00.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.373+0000 7ff747f92740 20 client.15342 trim_cache size 0 max 16384 2024-03-21T08:57:00.390 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 Processor -- start 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 -- start start 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 0x5582f265a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f26711c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 0x5582f265a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2675a90 con 0x5582f26656b0 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f26760f0 con 0x5582f2660540 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff747f92740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2676750 con 0x5582f265fb50 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 0x5582f265a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 0x5582f265a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff737fff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f26711c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 0x5582f265a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:51388/0 (socket says 172.21.15.87:51388) 2024-03-21T08:57:00.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 -- 172.21.15.87:0/179008432 learned_addr learned my addr 172.21.15.87:0/179008432 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73f7fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 0x5582f265a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59222/0 (socket says 172.21.15.87:59222) 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 -- 172.21.15.87:0/179008432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 msgr2=0x5582f26711c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f26711c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 -- 172.21.15.87:0/179008432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 msgr2=0x5582f265a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 0x5582f265a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73effd640 1 -- 172.21.15.87:0/179008432 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582f239f6b0 con 0x5582f265fb50 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.390+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 0x5582f265a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff73effd640 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 0x5582f265a2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7300bd6c0 tx=0x7ff730005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81790b4e5dfd10e server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff734ff9640 1 -- 172.21.15.87:0/179008432 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73004b070 con 0x5582f265fb50 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff734ff9640 1 -- 172.21.15.87:0/179008432 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff730004c80 con 0x5582f265fb50 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff734ff9640 1 -- 172.21.15.87:0/179008432 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300bd090 con 0x5582f265fb50 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 -- 172.21.15.87:0/179008432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 msgr2=0x5582f265a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 0x5582f265a2f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff7300bd6c0 tx=0x7ff730005fd0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 -- 172.21.15.87:0/179008432 shutdown_connections 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f26656b0 0x5582f265a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f26711c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 --2- 172.21.15.87:0/179008432 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f265fb50 0x5582f265a2f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 -- 172.21.15.87:0/179008432 >> 172.21.15.87:0/179008432 conn(0x5582f2623a60 msgr2=0x5582f2661a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 -- 172.21.15.87:0/179008432 shutdown_connections 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.391+0000 7ff747f92740 1 -- 172.21.15.87:0/179008432 wait complete. 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 -- start start 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 Processor -- start 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2670980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f2670ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f266d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2677410 con 0x5582f265fb50 2024-03-21T08:57:00.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2676db0 con 0x5582f2660540 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582f2677a70 con 0x5582f26656b0 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff747f92740 10 client.0 _ll_register_callbacks cb 0x7ff728009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2670980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2670980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59238/0 (socket says 172.21.15.87:59238) 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 -- 172.21.15.87:0/866199907 learned_addr learned my addr 172.21.15.87:0/866199907 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73f7fe640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f266d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff737fff640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f2670ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 -- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 msgr2=0x5582f266d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f266d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 -- 172.21.15.87:0/866199907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 msgr2=0x5582f2670ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f2670ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 -- 172.21.15.87:0/866199907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582f266da90 con 0x5582f265fb50 2024-03-21T08:57:00.394 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff73effd640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2670980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff730005ab0 tx=0x7ff7300bbc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b0018cd5adfed8 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.392+0000 7ff7267fc640 10 client.0 ms_handle_connect on v2:172.21.15.87:3300/0 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff7267fc640 1 -- 172.21.15.87:0/866199907 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73004b030 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff7267fc640 1 -- 172.21.15.87:0/866199907 <== mon.0 v2:172.21.15.87:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff7300bc920 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff730004c80 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff7267fc640 1 -- 172.21.15.87:0/866199907 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7300bd090 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 10 client.15348 fetch_fsmap learned FSMap version 25 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5582f266da90 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff7267fc640 1 -- 172.21.15.87:0/866199907 <== mon.0 v2:172.21.15.87:3300/0 4 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff730004c80 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff7267fc640 1 -- 172.21.15.87:0/866199907 <== mon.0 v2:172.21.15.87:3300/0 5 ==== fsmap(e 25) v1 ==== 7934+0+0 (secure 0 0 0) 0x7ff7300f4040 con 0x5582f265fb50 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 10 client.15348 fetch_fsmap finished waiting for FSMap version 25 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 10 client.15348 resolve_mds: resolved name 'i' to daemon mds.i 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 20 client.15348 populate_metadata read hostname 'smithi087' 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f26d5e90 0x5582f26d1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:00.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 4 client.15348 mds_command: new command op to 24487 tid=0 multi_id=0 [{"prefix": "status"}] 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 --> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5582f25c84b0 con 0x5582f26d5e90 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.393+0000 7ff737fff640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f26d5e90 0x5582f26d1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7267fc640 10 client.15348 ms_handle_connect on v2:172.21.15.105:6838/892092628 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff737fff640 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f26d5e90 0x5582f26d1ef0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e63d61db76d55a41 server_cookie=2891c6c0d2d8a20e in_seq=0 out_seq=0 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7267fc640 1 -- 172.21.15.87:0/866199907 <== mds.0 v2:172.21.15.105:6838/892092628 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5582f25c84b0 con 0x5582f26d5e90 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7267fc640 10 client.15348 handle_command_reply: tid=0 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 client.15348 shutdown 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 msgr2=0x5582f2670980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2670980 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff730005ab0 tx=0x7ff7300bbc40 comp rx=0 tx=0).stop 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 shutdown_connections 2024-03-21T08:57:00.396 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:6838/892092628,v1:172.21.15.105:6839/892092628] conn(0x5582f26d5e90 0x5582f26d1ef0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x5582f26656b0 0x5582f266d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x5582f2660540 0x5582f2670ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 --2- 172.21.15.87:0/866199907 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x5582f265fb50 0x5582f2670980 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 >> 172.21.15.87:0/866199907 conn(0x5582f2623a60 msgr2=0x5582f265a2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7267fc640 0 client.15348 ms_handle_reset on v2:172.21.15.105:6838/892092628 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 shutdown_connections 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 1 -- 172.21.15.87:0/866199907 wait complete. 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff747f92740 20 client.15348 trim_cache size 0 max 16384 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T08:57:00.397 INFO:teuthology.orchestra.run.smithi087.stdout: "cluster_fsid": "75a2f1f0-e75f-11ee-95cd-87774f69a715", 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "whoami": 0, 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "id": 24487, 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "want_state": "up:active", 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "state": "up:active", 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "fs_name": "cephfs", 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "rank_uptime": 112.872566414, 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "mdsmap_epoch": 25, 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch": 88, 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_epoch_barrier": 86, 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout: "uptime": 128.571088987 2024-03-21T08:57:00.398 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff72c0a2f20 msgr2=0x7ff72c0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7452c6640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff72c0a2f20 0x7ff72c0a53e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x5582f2660da0 tx=0x7ff738014040 comp rx=0 tx=0).stop 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.394+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 msgr2=0x7ff74006a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff74006a7e0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7ff730000c00 tx=0x7ff73002c640 comp rx=0 tx=0).stop 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 shutdown_connections 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff72c0a2f20 0x7ff72c0a53e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.399 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff740102170 0x7ff74019ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff7401017b0 0x7ff74019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 --2- 172.21.15.87:0/1993699771 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff7401005c0 0x7ff74006a7e0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:00.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 >> 172.21.15.87:0/1993699771 conn(0x7ff7400fbb50 msgr2=0x7ff740104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:00.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 shutdown_connections 2024-03-21T08:57:00.400 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:00.395+0000 7ff7452c6640 1 -- 172.21.15.87:0/1993699771 wait complete. 2024-03-21T08:57:00.401 INFO:teuthology.run_tasks:Running task kclient... 2024-03-21T08:57:00.425 INFO:tasks.kclient:Mounting kernel clients... 2024-03-21T08:57:00.425 INFO:tasks.kclient:config is {'client.0': {}} 2024-03-21T08:57:00.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:00.425 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-03-21T08:57:00.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:00.440 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-03-21T08:57:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:00 smithi105 ceph-mon[35993]: pgmap v348: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:00 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2415551941' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T08:57:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:00 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1381819186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T08:57:00.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:00 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2019517876' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T08:57:00.509 INFO:teuthology.orchestra.run.smithi087.stderr:Cannot find device "ceph-brx" 2024-03-21T08:57:00.511 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T08:57:00.512 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-03-21T08:57:00.512 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-03-21T08:57:00.512 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:00.512 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph fs ls 2024-03-21T08:57:00.790 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T08:57:02.237 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:02 smithi087 ceph-mon[32385]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:02.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:02 smithi100 ceph-mon[36306]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:02.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:02 smithi105 ceph-mon[35993]: pgmap v349: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:02.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 -- 172.21.15.87:0/589117842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4108290 msgr2=0x7fceb4108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:02.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 --2- 172.21.15.87:0/589117842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4108290 0x7fceb4108670 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fcea4009990 tx=0x7fcea4009ca0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.659 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 -- 172.21.15.87:0/589117842 shutdown_connections 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 --2- 172.21.15.87:0/589117842 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb4102ba0 0x7fceb410f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 --2- 172.21.15.87:0/589117842 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4108290 0x7fceb4108670 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 --2- 172.21.15.87:0/589117842 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb41021e0 0x7fceb4102660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 -- 172.21.15.87:0/589117842 >> 172.21.15.87:0/589117842 conn(0x7fceb40792c0 msgr2=0x7fceb40796d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 -- 172.21.15.87:0/589117842 shutdown_connections 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.663+0000 7fceb9b87700 1 -- 172.21.15.87:0/589117842 wait complete. 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 Processor -- start 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 -- start start 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb41021e0 0x7fceb419e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 0x7fceb419eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb4108290 0x7fceb4198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:02.660 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb41124a0 con 0x7fceb4102ba0 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb4112320 con 0x7fceb41021e0 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.664+0000 7fceb9b87700 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fceb4112620 con 0x7fceb4108290 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 0x7fceb419eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 0x7fceb419eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59254/0 (socket says 172.21.15.87:59254) 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 -- 172.21.15.87:0/2655960901 learned_addr learned my addr 172.21.15.87:0/2655960901 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb37fe700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb41021e0 0x7fceb419e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:02.661 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb3fff700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb4108290 0x7fceb4198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 -- 172.21.15.87:0/2655960901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb4108290 msgr2=0x7fceb4198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb4108290 0x7fceb4198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 -- 172.21.15.87:0/2655960901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb41021e0 msgr2=0x7fceb419e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb41021e0 0x7fceb419e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 -- 172.21.15.87:0/2655960901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceb4198f60 con 0x7fceb4102ba0 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb37fe700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb41021e0 0x7fceb419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb3fff700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb4108290 0x7fceb4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb2ffd700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 0x7fceb419eb40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcea800d980 tx=0x7fcea800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8016070 con 0x7fceb4102ba0 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fcea8004d10 con 0x7fceb4102ba0 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fceb4199250 con 0x7fceb4102ba0 2024-03-21T08:57:02.663 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcea8005670 con 0x7fceb4102ba0 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.665+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fceb41a5460 con 0x7fceb4102ba0 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.666+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fceb41038d0 con 0x7fceb4102ba0 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.667+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fcea8004750 con 0x7fceb4102ba0 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.668+0000 7fceb0ff9700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fce9c0a2fb0 0x7fce9c0a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.668+0000 7fceb37fe700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fce9c0a2fb0 0x7fce9c0a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.668+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fcea80cf150 con 0x7fceb4102ba0 2024-03-21T08:57:02.664 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.668+0000 7fceb37fe700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fce9c0a2fb0 0x7fce9c0a5470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098e0 tx=0x7fcea4005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:02.667 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.672+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fcea801f020 con 0x7fceb4102ba0 2024-03-21T08:57:02.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.915+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fceb402da30 con 0x7fceb4102ba0 2024-03-21T08:57:02.910 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.915+0000 7fceb0ff9700 1 -- 172.21.15.87:0/2655960901 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7fcea809bfb0 con 0x7fceb4102ba0 2024-03-21T08:57:02.910 INFO:teuthology.orchestra.run.smithi087.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-03-21T08:57:02.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fce9c0a2fb0 msgr2=0x7fce9c0a5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:02.912 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fce9c0a2fb0 0x7fce9c0a5470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fcea40098e0 tx=0x7fcea4005960 comp rx=0 tx=0).stop 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 msgr2=0x7fceb419eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 0x7fceb419eb40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fcea800d980 tx=0x7fcea800dc90 comp rx=0 tx=0).stop 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 shutdown_connections 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fce9c0a2fb0 0x7fce9c0a5470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fceb41021e0 0x7fceb419e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fceb4108290 0x7fceb4198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 --2- 172.21.15.87:0/2655960901 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fceb4102ba0 0x7fceb419eb40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 >> 172.21.15.87:0/2655960901 conn(0x7fceb40792c0 msgr2=0x7fceb4100e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 shutdown_connections 2024-03-21T08:57:02.913 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:02.917+0000 7fceb9b87700 1 -- 172.21.15.87:0/2655960901 wait complete. 2024-03-21T08:57:03.168 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:03 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2655960901' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:57:03.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:03 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2655960901' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:57:03.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:03 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2655960901' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi087.front.sepia.ceph.com 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-03-21T08:57:03.700 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-03-21T08:57:03.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:03.701 DEBUG:teuthology.orchestra.run.smithi087:> ip addr 2024-03-21T08:57:03.717 INFO:teuthology.orchestra.run.smithi087.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-03-21T08:57:03.717 INFO:teuthology.orchestra.run.smithi087.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: inet 127.0.0.1/8 scope host lo 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 ::1/128 scope host 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fc brd ff:ff:ff:ff:ff:ff 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f0 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:57:03.718 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:d8:da:fd brd ff:ff:ff:ff:ff:ff 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: altname enp7s0f1 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f4 brd ff:ff:ff:ff:ff:ff 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: link/ether 0c:c4:7a:8f:d5:f5 brd ff:ff:ff:ff:ff:ff 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: inet 172.21.15.87/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft 42090sec preferred_lft 42090sec 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: inet6 fe80::ec4:7aff:fe8f:d5f5/64 scope link 2024-03-21T08:57:03.719 INFO:teuthology.orchestra.run.smithi087.stdout: valid_lft forever preferred_lft forever 2024-03-21T08:57:03.720 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-03-21T08:57:03.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:03.720 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-03-21T08:57:03.720 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-03-21T08:57:03.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add name ceph-brx type bridge 2024-03-21T08:57:03.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr flush dev ceph-brx 2024-03-21T08:57:03.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set ceph-brx up 2024-03-21T08:57:03.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-03-21T08:57:03.721 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-03-21T08:57:03.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:03.880 DEBUG:teuthology.orchestra.run.smithi087:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-03-21T08:57:03.907 INFO:teuthology.orchestra.run.smithi087.stdout:1 2024-03-21T08:57:03.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:03.909 DEBUG:teuthology.orchestra.run.smithi087:> route 2024-03-21T08:57:03.997 INFO:teuthology.orchestra.run.smithi087.stdout:Kernel IP routing table 2024-03-21T08:57:03.997 INFO:teuthology.orchestra.run.smithi087.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-03-21T08:57:03.997 INFO:teuthology.orchestra.run.smithi087.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-03-21T08:57:03.997 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-03-21T08:57:03.997 INFO:teuthology.orchestra.run.smithi087.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-03-21T08:57:03.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:03.998 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-03-21T08:57:03.998 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-03-21T08:57:03.998 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-03-21T08:57:03.998 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-03-21T08:57:03.998 DEBUG:teuthology.orchestra.run.smithi087:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-03-21T08:57:03.998 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-03-21T08:57:04.240 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:04 smithi087 ceph-mon[32385]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:04.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.241 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-03-21T08:57:04.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.256 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list-id 2024-03-21T08:57:04.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:04 smithi100 ceph-mon[36306]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:04.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.312 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-03-21T08:57:04.312 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-03-21T08:57:04.312 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T08:57:04.312 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-03-21T08:57:04.312 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-03-21T08:57:04.394 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-03-21T08:57:04.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.394 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-03-21T08:57:04.395 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-03-21T08:57:04.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:04 smithi105 ceph-mon[35993]: pgmap v350: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:04.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.535 DEBUG:teuthology.orchestra.run.smithi087:> (cd / && exec bash -c ' 2024-03-21T08:57:04.535 DEBUG:teuthology.orchestra.run.smithi087:> set -e 2024-03-21T08:57:04.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set brx.0 up 2024-03-21T08:57:04.535 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link set dev brx.0 master ceph-brx 2024-03-21T08:57:04.535 DEBUG:teuthology.orchestra.run.smithi087:> ') 2024-03-21T08:57:04.577 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T08:57:04.577 DEBUG:teuthology.orchestra.run.smithi087:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:04.635 INFO:teuthology.orchestra.run.smithi087.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-03-21T08:57:04.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-03-21T08:57:04.636 DEBUG:teuthology.orchestra.run.smithi087:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:04.691 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-03-21T08:57:04.691 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-03-21T08:57:04.691 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/9460823c-257e-4c9f-be0e-ef702a13bc5b 2024-03-21T08:57:04.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.692 DEBUG:teuthology.orchestra.run.smithi087:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/9460823c-257e-4c9f-be0e-ef702a13bc5b /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-03-21T08:57:04.776 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-03-21T08:57:04.776 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-03-21T08:57:04.898 INFO:teuthology.orchestra.run.smithi087.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-03-21T08:57:04.898 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-03-21T08:57:04.898 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: resolved to: "172.21.15.87:3300,172.21.15.100:3300,172.21.15.105:3300" 2024-03-21T08:57:04.898 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: trying mount with new device syntax: 0@75a2f1f0-e75f-11ee-95cd-87774f69a715.cephfs=/volumes/qa/sv_1/9460823c-257e-4c9f-be0e-ef702a13bc5b 2024-03-21T08:57:04.898 INFO:teuthology.orchestra.run.smithi087.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.87:3300/172.21.15.100:3300/172.21.15.105:3300" will pass to kernel 2024-03-21T08:57:04.899 INFO:teuthology.orchestra.run.smithi087.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-03-21T08:57:04.899 INFO:teuthology.orchestra.run.smithi087.stdout: You just mounted a file system that supports labels which does not 2024-03-21T08:57:04.899 INFO:teuthology.orchestra.run.smithi087.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-03-21T08:57:04.899 INFO:teuthology.orchestra.run.smithi087.stdout: applications will generate AVC messages and not be allowed access to 2024-03-21T08:57:04.899 INFO:teuthology.orchestra.run.smithi087.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-03-21T08:57:04.901 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-03-21T08:57:04.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:04.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:04.932 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T08:57:04.989 INFO:teuthology.orchestra.run.smithi087.stdout:client15357 2024-03-21T08:57:04.989 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:57:04.989 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T08:57:05.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:05.047 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T08:57:05.103 INFO:teuthology.orchestra.run.smithi087.stdout:client15357 2024-03-21T08:57:05.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:05.104 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-03-21T08:57:05.160 INFO:teuthology.orchestra.run.smithi087.stdout:client15357 2024-03-21T08:57:05.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T08:57:05.160 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715.client15357/status') 2024-03-21T08:57:05.227 INFO:teuthology.orchestra.run.smithi087.stdout:instance: client.15357 (3)192.168.0.1:0/1406986390 2024-03-21T08:57:05.227 INFO:teuthology.orchestra.run.smithi087.stdout:blocklisted: false 2024-03-21T08:57:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records in 2024-03-21T08:57:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:0+1 records out 2024-03-21T08:57:05.228 INFO:teuthology.orchestra.run.smithi087.stderr:70 bytes copied, 2.0761e-05 s, 3.4 MB/s 2024-03-21T08:57:05.229 INFO:teuthology.run_tasks:Running task exec... 2024-03-21T08:57:05.244 INFO:teuthology.task.exec:Executing custom commands... 2024-03-21T08:57:05.244 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi087.front.sepia.ceph.com 2024-03-21T08:57:05.244 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-03-21T08:57:05.373 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.376+0000 7f173c2f8640 1 -- 172.21.15.87:0/1696663016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734102170 msgr2=0x7f1734108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- 172.21.15.87:0/1696663016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734102170 0x7f1734108a00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f172c009ca0 tx=0x7f172c02c0f0 comp rx=0 tx=0).stop 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- 172.21.15.87:0/1696663016 shutdown_connections 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- 172.21.15.87:0/1696663016 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1734102170 0x7f1734108a00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- 172.21.15.87:0/1696663016 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17341017b0 0x7f1734101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- 172.21.15.87:0/1696663016 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17341005c0 0x7f17341009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- 172.21.15.87:0/1696663016 >> 172.21.15.87:0/1696663016 conn(0x7f17340fbb30 msgr2=0x7f17340fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- 172.21.15.87:0/1696663016 shutdown_connections 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- 172.21.15.87:0/1696663016 wait complete. 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 Processor -- start 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- start start 2024-03-21T08:57:05.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17341005c0 0x7f1734197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 0x7f1734197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1734102170 0x7f173419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173410d360 con 0x7f17341017b0 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173410d1e0 con 0x7f1734102170 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f173c2f8640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173410d4e0 con 0x7f17341005c0 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f1739893640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 0x7f1734197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f1739893640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 0x7f1734197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59268/0 (socket says 172.21.15.87:59268) 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.377+0000 7f1739893640 1 -- 172.21.15.87:0/130199614 learned_addr learned my addr 172.21.15.87:0/130199614 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f173a094640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17341005c0 0x7f1734197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f173a895640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1734102170 0x7f173419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f1739893640 1 -- 172.21.15.87:0/130199614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17341005c0 msgr2=0x7f1734197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f1739893640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17341005c0 0x7f1734197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:05.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f1739893640 1 -- 172.21.15.87:0/130199614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1734102170 msgr2=0x7f173419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f1739893640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1734102170 0x7f173419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f1739893640 1 -- 172.21.15.87:0/130199614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f173419f530 con 0x7f17341017b0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f1739893640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 0x7f1734197ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f172400e9f0 tx=0x7f172400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172400cd80 con 0x7f17341017b0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f173419f820 con 0x7f17341017b0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1724005500 con 0x7f17341017b0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724010690 con 0x7f17341017b0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.378+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173419fd20 con 0x7f17341017b0 2024-03-21T08:57:05.376 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.379+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1734107810 con 0x7f17341017b0 2024-03-21T08:57:05.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.380+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f17240046c0 con 0x7f17341017b0 2024-03-21T08:57:05.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.380+0000 7f172b7fe640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17200a2fc0 0x7f17200a5480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:05.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.381+0000 7f173a094640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17200a2fc0 0x7f17200a5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:05.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.381+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f17240cf7b0 con 0x7f17341017b0 2024-03-21T08:57:05.377 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.381+0000 7f173a094640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17200a2fc0 0x7f17200a5480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1734101610 tx=0x7f1730005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:05.380 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.385+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f172409c610 con 0x7f17341017b0 2024-03-21T08:57:05.599 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:05.605+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) v1 -- 0x7f17340622a0 con 0x7f17341017b0 2024-03-21T08:57:06.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.020+0000 7f172b7fe640 1 -- 172.21.15.87:0/130199614 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) v1 ==== 118+0+0 (secure 0 0 0) 0x7f172409bf10 con 0x7f17341017b0 2024-03-21T08:57:06.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17200a2fc0 msgr2=0x7f17200a5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.018 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17200a2fc0 0x7f17200a5480 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1734101610 tx=0x7f1730005f90 comp rx=0 tx=0).stop 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 msgr2=0x7f1734197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 0x7f1734197ea0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f172400e9f0 tx=0x7f172400edf0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 shutdown_connections 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f17200a2fc0 0x7f17200a5480 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1734102170 0x7f173419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17341017b0 0x7f1734197ea0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 --2- 172.21.15.87:0/130199614 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17341005c0 0x7f1734197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 >> 172.21.15.87:0/130199614 conn(0x7f17340fbb30 msgr2=0x7f17341049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 shutdown_connections 2024-03-21T08:57:06.019 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.023+0000 7f173c2f8640 1 -- 172.21.15.87:0/130199614 wait complete. 2024-03-21T08:57:06.030 DEBUG:teuthology.orchestra.run.smithi087:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 -- 172.21.15.87:0/1599270034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398107450 msgr2=0x7ff398109840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/1599270034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398107450 0x7ff398109840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7ff390009ca0 tx=0x7ff39002c0f0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 -- 172.21.15.87:0/1599270034 shutdown_connections 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/1599270034 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff398107450 0x7ff398109840 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/1599270034 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff398104b20 0x7ff398106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/1599270034 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff3981021f0 0x7ff3981045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 -- 172.21.15.87:0/1599270034 >> 172.21.15.87:0/1599270034 conn(0x7ff3980fbb80 msgr2=0x7ff3980fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:06.139 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 -- 172.21.15.87:0/1599270034 shutdown_connections 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.141+0000 7ff39dcc0640 1 -- 172.21.15.87:0/1599270034 wait complete. 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 Processor -- start 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 -- start start 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 0x7ff398197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff398104b20 0x7ff398197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff398107450 0x7ff39819ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39810d3b0 con 0x7ff3981021f0 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39810d230 con 0x7ff398104b20 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff39dcc0640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff39810d530 con 0x7ff398107450 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 0x7ff398197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff3977fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 0x7ff398197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:59288/0 (socket says 172.21.15.87:59288) 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff3977fe640 1 -- 172.21.15.87:0/2256180204 learned_addr learned my addr 172.21.15.87:0/2256180204 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T08:57:06.140 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff38effd640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff398104b20 0x7ff398197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.142+0000 7ff397fff640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff398107450 0x7ff39819ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3977fe640 1 -- 172.21.15.87:0/2256180204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff398107450 msgr2=0x7ff39819ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3977fe640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff398107450 0x7ff39819ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3977fe640 1 -- 172.21.15.87:0/2256180204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff398104b20 msgr2=0x7ff398197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3977fe640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff398104b20 0x7ff398197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3977fe640 1 -- 172.21.15.87:0/2256180204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39819f500 con 0x7ff3981021f0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff397fff640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff398107450 0x7ff39819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3977fe640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 0x7ff398197930 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff38800e9f0 tx=0x7ff38800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38800cd80 con 0x7ff3981021f0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7ff388005500 con 0x7ff3981021f0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff39819f7f0 con 0x7ff3981021f0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff388010690 con 0x7ff3981021f0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.143+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff39819fd40 con 0x7ff3981021f0 2024-03-21T08:57:06.141 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.144+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3980fd670 con 0x7ff3981021f0 2024-03-21T08:57:06.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.145+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 18) v1 ==== 137854+0+0 (secure 0 0 0) 0x7ff3880046c0 con 0x7ff3981021f0 2024-03-21T08:57:06.142 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.145+0000 7ff3957fa640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff3840a3040 0x7ff3840a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T08:57:06.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.146+0000 7ff38effd640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff3840a3040 0x7ff3840a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T08:57:06.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.146+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7ff3880cf830 con 0x7ff3981021f0 2024-03-21T08:57:06.143 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.146+0000 7ff38effd640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff3840a3040 0x7ff3840a5500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff398198df0 tx=0x7ff380005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T08:57:06.145 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.150+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7ff38809c630 con 0x7ff3981021f0 2024-03-21T08:57:06.351 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.356+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 --> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) v1 -- 0x7ff398061b80 con 0x7ff3840a3040 2024-03-21T08:57:06.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-mon[32385]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:06.358 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/130199614' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T08:57:06.372 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.377+0000 7ff3957fa640 1 -- 172.21.15.87:0/2256180204 <== mgr.14152 v2:172.21.15.87:6800/1005139006 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3 (secure 0 0 0) 0x7ff398061b80 con 0x7ff3840a3040 2024-03-21T08:57:06.372 INFO:teuthology.orchestra.run.smithi087.stdout:{} 2024-03-21T08:57:06.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff3840a3040 msgr2=0x7ff3840a5500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff3840a3040 0x7ff3840a5500 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff398198df0 tx=0x7ff380005f90 comp rx=0 tx=0).stop 2024-03-21T08:57:06.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 msgr2=0x7ff398197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T08:57:06.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 0x7ff398197930 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ff38800e9f0 tx=0x7ff38800edf0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 shutdown_connections 2024-03-21T08:57:06.374 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7ff3840a3040 0x7ff3840a5500 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7ff398107450 0x7ff39819ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7ff398104b20 0x7ff398197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 --2- 172.21.15.87:0/2256180204 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7ff3981021f0 0x7ff398197930 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T08:57:06.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 >> 172.21.15.87:0/2256180204 conn(0x7ff3980fbb80 msgr2=0x7ff398102b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T08:57:06.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 shutdown_connections 2024-03-21T08:57:06.375 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T08:57:06.379+0000 7ff39dcc0640 1 -- 172.21.15.87:0/2256180204 wait complete. 2024-03-21T08:57:06.385 INFO:teuthology.run_tasks:Running task check-counter... 2024-03-21T08:57:06.398 INFO:teuthology.run_tasks:Running task workunit... 2024-03-21T08:57:06.457 INFO:tasks.workunit:Pulling workunits from ref 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:57:06.457 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-21T08:57:06.457 INFO:tasks.workunit:timeout=3h 2024-03-21T08:57:06.457 INFO:tasks.workunit:cleanup=True 2024-03-21T08:57:06.458 DEBUG:teuthology.orchestra.run.smithi087:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:06.473 INFO:teuthology.orchestra.run.smithi087.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:06.473 INFO:teuthology.orchestra.run.smithi087.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-03-21T08:57:06.473 INFO:teuthology.orchestra.run.smithi087.stdout:Device: 77h/119d Inode: 1099511628283 Links: 2 2024-03-21T08:57:06.474 INFO:teuthology.orchestra.run.smithi087.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-21T08:57:06.474 INFO:teuthology.orchestra.run.smithi087.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-03-21T08:57:06.474 INFO:teuthology.orchestra.run.smithi087.stdout:Access: 2024-03-21 08:55:12.984367184 +0000 2024-03-21T08:57:06.474 INFO:teuthology.orchestra.run.smithi087.stdout:Modify: 2024-03-21 08:55:12.984367184 +0000 2024-03-21T08:57:06.474 INFO:teuthology.orchestra.run.smithi087.stdout:Change: 2024-03-21 08:57:04.932292125 +0000 2024-03-21T08:57:06.474 INFO:teuthology.orchestra.run.smithi087.stdout: Birth: 2024-03-21 08:55:12.984367184 +0000 2024-03-21T08:57:06.474 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-03-21T08:57:06.474 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-03-21T08:57:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:06 smithi105 ceph-mon[35993]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:06 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/130199614' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T08:57:06.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:06 smithi100 ceph-mon[36306]: pgmap v351: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:06.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:06 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/130199614' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-03-21T08:57:06.548 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 2edfee68ad7243bed422a049b6f5f58619300f11 2024-03-21T08:57:06.603 INFO:tasks.workunit.client.0.smithi087.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-21T08:57:06.656 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:57:06.360+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:57:06.656 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:57:06.360+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:57:06.656 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:57:06.360+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:57:06.656 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:57:06.360+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:57:06.656 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 08:57:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T08:57:06.360+0000 7fbb6767e700 -1 client.0 error registering admin socket command: (17) File exists 2024-03-21T08:57:07.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:07 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/130199614' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T08:57:07.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:07 smithi087 ceph-mon[32385]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T08:57:07.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:07 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/130199614' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T08:57:07.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:07 smithi105 ceph-mon[35993]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T08:57:07.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:07 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/130199614' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-03-21T08:57:07.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:07 smithi100 ceph-mon[36306]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-03-21T08:57:08.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:08 smithi087 ceph-mon[32385]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:57:08.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:08 smithi087 ceph-mon[32385]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:08.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:08 smithi105 ceph-mon[35993]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:57:08.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:08 smithi105 ceph-mon[35993]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:08 smithi100 ceph-mon[36306]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-03-21T08:57:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:08 smithi100 ceph-mon[36306]: pgmap v352: 97 pgs: 97 active+clean; 643 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:09.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:09 smithi087 ceph-mon[32385]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-21T08:57:09.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:09 smithi087 ceph-mon[32385]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-21T08:57:09.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:09 smithi105 ceph-mon[35993]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-21T08:57:09.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:09 smithi105 ceph-mon[35993]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-21T08:57:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:09 smithi100 ceph-mon[36306]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-03-21T08:57:09.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:09 smithi100 ceph-mon[36306]: pgmap v353: 97 pgs: 97 active+clean; 645 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 11 op/s 2024-03-21T08:57:12.185 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:11 smithi105 ceph-mon[35993]: pgmap v354: 97 pgs: 97 active+clean; 657 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-03-21T08:57:12.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:11 smithi100 ceph-mon[36306]: pgmap v354: 97 pgs: 97 active+clean; 657 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-03-21T08:57:12.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:11 smithi087 ceph-mon[32385]: pgmap v354: 97 pgs: 97 active+clean; 657 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-03-21T08:57:14.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:13 smithi105 ceph-mon[35993]: pgmap v355: 97 pgs: 97 active+clean; 657 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-03-21T08:57:14.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:13 smithi100 ceph-mon[36306]: pgmap v355: 97 pgs: 97 active+clean; 657 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-03-21T08:57:14.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:13 smithi087 ceph-mon[32385]: pgmap v355: 97 pgs: 97 active+clean; 657 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 2.3 KiB/s wr, 12 op/s 2024-03-21T08:57:16.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:15 smithi105 ceph-mon[35993]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-03-21T08:57:16.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:15 smithi100 ceph-mon[36306]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-03-21T08:57:16.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:15 smithi087 ceph-mon[32385]: pgmap v356: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-03-21T08:57:18.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:17 smithi105 ceph-mon[35993]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-21T08:57:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:17 smithi100 ceph-mon[36306]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-21T08:57:18.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:17 smithi087 ceph-mon[32385]: pgmap v357: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 2.5 KiB/s wr, 10 op/s 2024-03-21T08:57:20.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:19 smithi105 ceph-mon[35993]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-03-21T08:57:20.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:19 smithi100 ceph-mon[36306]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-03-21T08:57:20.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:19 smithi087 ceph-mon[32385]: pgmap v358: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.5 KiB/s wr, 13 op/s 2024-03-21T08:57:22.208 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:21 smithi087 ceph-mon[32385]: pgmap v359: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T08:57:22.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:21 smithi105 ceph-mon[35993]: pgmap v359: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T08:57:22.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:21 smithi100 ceph-mon[36306]: pgmap v359: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-03-21T08:57:24.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:23 smithi105 ceph-mon[35993]: pgmap v360: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:57:24.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:23 smithi100 ceph-mon[36306]: pgmap v360: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:57:24.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:23 smithi087 ceph-mon[32385]: pgmap v360: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:57:26.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:25 smithi105 ceph-mon[35993]: pgmap v361: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:57:26.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:25 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:57:26.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:25 smithi100 ceph-mon[36306]: pgmap v361: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:57:26.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:25 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:57:26.341 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:25 smithi087 ceph-mon[32385]: pgmap v361: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 170 B/s wr, 10 op/s 2024-03-21T08:57:26.341 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:25 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:57:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: pgmap v362: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:57:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:57:28.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:27 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: pgmap v362: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:57:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:57:28.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:27 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: pgmap v362: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:57:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:57:28.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:27 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:57:30.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:29 smithi105 ceph-mon[35993]: pgmap v363: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:30.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:29 smithi100 ceph-mon[36306]: pgmap v363: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:30.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:29 smithi087 ceph-mon[32385]: pgmap v363: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:32.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:31 smithi105 ceph-mon[35993]: pgmap v364: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:32.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:31 smithi100 ceph-mon[36306]: pgmap v364: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:32.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:31 smithi087 ceph-mon[32385]: pgmap v364: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:34.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:33 smithi105 ceph-mon[35993]: pgmap v365: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:34.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:33 smithi100 ceph-mon[36306]: pgmap v365: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:34.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:33 smithi087 ceph-mon[32385]: pgmap v365: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:36.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:35 smithi105 ceph-mon[35993]: pgmap v366: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:36.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:35 smithi100 ceph-mon[36306]: pgmap v366: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:36.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:35 smithi087 ceph-mon[32385]: pgmap v366: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:57:38.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:37 smithi105 ceph-mon[35993]: pgmap v367: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:38.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:37 smithi100 ceph-mon[36306]: pgmap v367: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:38.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:37 smithi087 ceph-mon[32385]: pgmap v367: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:40.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:39 smithi105 ceph-mon[35993]: pgmap v368: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:40.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:39 smithi100 ceph-mon[36306]: pgmap v368: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:40.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:39 smithi087 ceph-mon[32385]: pgmap v368: 97 pgs: 97 active+clean; 659 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:57:42.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:41 smithi105 ceph-mon[35993]: pgmap v369: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:57:42.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:41 smithi100 ceph-mon[36306]: pgmap v369: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:57:42.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:41 smithi087 ceph-mon[32385]: pgmap v369: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:57:44.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:43 smithi105 ceph-mon[35993]: pgmap v370: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:57:44.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:43 smithi100 ceph-mon[36306]: pgmap v370: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:57:44.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:43 smithi087 ceph-mon[32385]: pgmap v370: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 10 op/s 2024-03-21T08:57:46.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:45 smithi105 ceph-mon[35993]: pgmap v371: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:46.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:45 smithi100 ceph-mon[36306]: pgmap v371: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:46.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:45 smithi087 ceph-mon[32385]: pgmap v371: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:48.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:47 smithi105 ceph-mon[35993]: pgmap v372: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-21T08:57:48.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:47 smithi100 ceph-mon[36306]: pgmap v372: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-21T08:57:48.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:47 smithi087 ceph-mon[32385]: pgmap v372: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 682 B/s wr, 8 op/s 2024-03-21T08:57:50.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:49 smithi105 ceph-mon[35993]: pgmap v373: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:50.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:49 smithi100 ceph-mon[36306]: pgmap v373: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:50.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:49 smithi087 ceph-mon[32385]: pgmap v373: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:52.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:51 smithi105 ceph-mon[35993]: pgmap v374: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:52.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:51 smithi100 ceph-mon[36306]: pgmap v374: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:52.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:51 smithi087 ceph-mon[32385]: pgmap v374: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 682 B/s wr, 10 op/s 2024-03-21T08:57:54.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:53 smithi105 ceph-mon[35993]: pgmap v375: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T08:57:54.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:53 smithi100 ceph-mon[36306]: pgmap v375: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T08:57:54.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:53 smithi087 ceph-mon[32385]: pgmap v375: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T08:57:56.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:55 smithi105 ceph-mon[35993]: pgmap v376: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T08:57:56.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:55 smithi100 ceph-mon[36306]: pgmap v376: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T08:57:56.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:55 smithi087 ceph-mon[32385]: pgmap v376: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 85 B/s wr, 10 op/s 2024-03-21T08:57:58.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:57 smithi100 ceph-mon[36306]: pgmap v377: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:58.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:57 smithi087 ceph-mon[32385]: pgmap v377: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:57:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:57 smithi105 ceph-mon[35993]: pgmap v377: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:00.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:57:59 smithi100 ceph-mon[36306]: pgmap v378: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:00.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:57:59 smithi087 ceph-mon[32385]: pgmap v378: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:57:59 smithi105 ceph-mon[35993]: pgmap v378: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:02.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:01 smithi100 ceph-mon[36306]: pgmap v379: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:02.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:01 smithi087 ceph-mon[32385]: pgmap v379: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:02.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:01 smithi105 ceph-mon[35993]: pgmap v379: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:04.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:03 smithi100 ceph-mon[36306]: pgmap v380: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:04.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:03 smithi087 ceph-mon[32385]: pgmap v380: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:04.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:03 smithi105 ceph-mon[35993]: pgmap v380: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:06.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:05 smithi100 ceph-mon[36306]: pgmap v381: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:06.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:05 smithi087 ceph-mon[32385]: pgmap v381: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:05 smithi105 ceph-mon[35993]: pgmap v381: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:08.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:07 smithi100 ceph-mon[36306]: pgmap v382: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:08.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:07 smithi087 ceph-mon[32385]: pgmap v382: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:08.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:07 smithi105 ceph-mon[35993]: pgmap v382: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:10.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:10 smithi100 ceph-mon[36306]: pgmap v383: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:10.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:10 smithi087 ceph-mon[32385]: pgmap v383: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:10.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:10 smithi105 ceph-mon[35993]: pgmap v383: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:12.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:12 smithi100 ceph-mon[36306]: pgmap v384: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:12.283 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:12 smithi105 ceph-mon[35993]: pgmap v384: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:12.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:12 smithi087 ceph-mon[32385]: pgmap v384: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:14.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:14 smithi100 ceph-mon[36306]: pgmap v385: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:14.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:14 smithi087 ceph-mon[32385]: pgmap v385: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:14.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:14 smithi105 ceph-mon[35993]: pgmap v385: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:16.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:16 smithi100 ceph-mon[36306]: pgmap v386: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:16.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:16 smithi087 ceph-mon[32385]: pgmap v386: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:16.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:16 smithi105 ceph-mon[35993]: pgmap v386: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:18 smithi100 ceph-mon[36306]: pgmap v387: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:18.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:18 smithi087 ceph-mon[32385]: pgmap v387: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:18.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:18 smithi105 ceph-mon[35993]: pgmap v387: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-03-21T08:58:20.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:20 smithi100 ceph-mon[36306]: pgmap v388: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:20.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:20 smithi087 ceph-mon[32385]: pgmap v388: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:20.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:20 smithi105 ceph-mon[35993]: pgmap v388: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:22.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:22 smithi100 ceph-mon[36306]: pgmap v389: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:22.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:22 smithi087 ceph-mon[32385]: pgmap v389: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:22.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:22 smithi105 ceph-mon[35993]: pgmap v389: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:24.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:24 smithi100 ceph-mon[36306]: pgmap v390: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:24.278 INFO:tasks.workunit.client.0.smithi087.stderr:Note: switching to '2edfee68ad7243bed422a049b6f5f58619300f11'. 2024-03-21T08:58:24.278 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.278 INFO:tasks.workunit.client.0.smithi087.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-21T08:58:24.278 INFO:tasks.workunit.client.0.smithi087.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr:state without impacting any branches by switching back to a branch. 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: git switch -c 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr:Or undo this operation with: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: git switch - 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:24.279 INFO:tasks.workunit.client.0.smithi087.stderr:HEAD is now at 2edfee68ad7 Merge PR #55768 into wip-vshankar-testing-20240320.053002 2024-03-21T08:58:24.283 DEBUG:teuthology.orchestra.run.smithi087:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-21T08:58:24.346 INFO:tasks.workunit.client.0.smithi087.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-21T08:58:24.348 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T08:58:24.348 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-21T08:58:24.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:24 smithi087 ceph-mon[32385]: pgmap v390: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:24.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:24 smithi105 ceph-mon[35993]: pgmap v390: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-03-21T08:58:25.078 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-21T08:58:25.138 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-21T08:58:25.177 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-21T08:58:25.179 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T08:58:25.179 INFO:tasks.workunit.client.0.smithi087.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-21T08:58:25.217 INFO:tasks.workunit.client.0.smithi087.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-21T08:58:25.220 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T08:58:25.220 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-21T08:58:25.276 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-03-21T08:58:25.277 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-03-21T08:58:25.277 DEBUG:teuthology.orchestra.run.smithi087:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=2edfee68ad7243bed422a049b6f5f58619300f11 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-03-21T08:58:25.338 INFO:tasks.workunit.client.0.smithi087.stdout:getting iogen 2024-03-21T08:58:25.339 INFO:tasks.workunit.client.0.smithi087.stderr:+ echo 'getting iogen' 2024-03-21T08:58:25.339 INFO:tasks.workunit.client.0.smithi087.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-03-21T08:58:25.362 INFO:tasks.workunit.client.0.smithi087.stderr:--2024-03-21 08:58:25-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-03-21T08:58:25.363 INFO:tasks.workunit.client.0.smithi087.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-03-21T08:58:25.412 INFO:tasks.workunit.client.0.smithi087.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-03-21T08:58:25.446 INFO:tasks.workunit.client.0.smithi087.stderr:HTTP request sent, awaiting response... 200 OK 2024-03-21T08:58:25.446 INFO:tasks.workunit.client.0.smithi087.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-03-21T08:58:25.447 INFO:tasks.workunit.client.0.smithi087.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-03-21T08:58:25.449 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:25.450 INFO:tasks.workunit.client.0.smithi087.stderr: 0K ....... 100% 344M=0s 2024-03-21T08:58:25.450 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:25.450 INFO:tasks.workunit.client.0.smithi087.stderr:2024-03-21 08:58:25 (344 MB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-03-21T08:58:25.450 INFO:tasks.workunit.client.0.smithi087.stderr: 2024-03-21T08:58:25.450 INFO:tasks.workunit.client.0.smithi087.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-03-21T08:58:25.462 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0 2024-03-21T08:58:25.464 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0/src 2024-03-21T08:58:25.465 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0/src/iogen.8 2024-03-21T08:58:25.466 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0/src/iogen.c 2024-03-21T08:58:25.467 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0/README 2024-03-21T08:58:25.468 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0/Makefile.bsd 2024-03-21T08:58:25.468 INFO:tasks.workunit.client.0.smithi087.stdout:iogen_3.1p0/Makefile 2024-03-21T08:58:25.469 INFO:tasks.workunit.client.0.smithi087.stderr:+ cd iogen_3.1p0 2024-03-21T08:58:25.469 INFO:tasks.workunit.client.0.smithi087.stdout:making iogen 2024-03-21T08:58:25.470 INFO:tasks.workunit.client.0.smithi087.stderr:+ echo 'making iogen' 2024-03-21T08:58:25.470 INFO:tasks.workunit.client.0.smithi087.stderr:+ make 2024-03-21T08:58:25.478 INFO:tasks.workunit.client.0.smithi087.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-03-21T08:58:25.704 INFO:tasks.workunit.client.0.smithi087.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-03-21T08:58:25.805 INFO:tasks.workunit.client.0.smithi087.stdout:running iogen 2024-03-21T08:58:25.806 INFO:tasks.workunit.client.0.smithi087.stderr:+ echo 'running iogen' 2024-03-21T08:58:25.806 INFO:tasks.workunit.client.0.smithi087.stderr:+ ./iogen -n 5 -s 2g 2024-03-21T08:58:25.806 INFO:tasks.workunit.client.0.smithi087.stderr:child spawned 84943 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stderr:child spawned 84944 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stderr:child spawned 84945 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stderr:child spawned 84946 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stderr:child spawned 84947 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stderr:+ echo 'sleep for 10 min' 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stdout:sleep for 10 min 2024-03-21T08:58:25.807 INFO:tasks.workunit.client.0.smithi087.stderr:+ sleep 600 2024-03-21T08:58:26.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:26 smithi087 ceph-mon[32385]: pgmap v391: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:26.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:26 smithi100 ceph-mon[36306]: pgmap v391: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:26.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:26 smithi105 ceph-mon[35993]: pgmap v391: 97 pgs: 97 active+clean; 664 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-03-21T08:58:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:28 smithi087 ceph-mon[32385]: pgmap v392: 97 pgs: 97 active+clean; 33 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 9 op/s 2024-03-21T08:58:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:28 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:58:28.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:28 smithi105 ceph-mon[35993]: pgmap v392: 97 pgs: 97 active+clean; 33 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 9 op/s 2024-03-21T08:58:28.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:28 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:58:28.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:28 smithi100 ceph-mon[36306]: pgmap v392: 97 pgs: 97 active+clean; 33 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.7 MiB/s wr, 9 op/s 2024-03-21T08:58:28.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:28 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:58:30.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:30 smithi087 ceph-mon[32385]: pgmap v393: 97 pgs: 97 active+clean; 601 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 50 MiB/s wr, 23 op/s 2024-03-21T08:58:30.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:30 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:30 smithi105 ceph-mon[35993]: pgmap v393: 97 pgs: 97 active+clean; 601 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 50 MiB/s wr, 23 op/s 2024-03-21T08:58:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:30 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:30 smithi100 ceph-mon[36306]: pgmap v393: 97 pgs: 97 active+clean; 601 MiB data, 420 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 50 MiB/s wr, 23 op/s 2024-03-21T08:58:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:30 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:31.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:58:31.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:58:31.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:31.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:58:31.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:58:31.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:31.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:58:31.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:58:31.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:58:32.286 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:32 smithi087 ceph-mon[32385]: pgmap v394: 97 pgs: 97 active+clean; 1.2 GiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 104 MiB/s wr, 111 op/s 2024-03-21T08:58:32.286 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:32 smithi087 ceph-mon[32385]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-21T08:58:32.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:32 smithi105 ceph-mon[35993]: pgmap v394: 97 pgs: 97 active+clean; 1.2 GiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 104 MiB/s wr, 111 op/s 2024-03-21T08:58:32.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:32 smithi105 ceph-mon[35993]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-21T08:58:32.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:32 smithi100 ceph-mon[36306]: pgmap v394: 97 pgs: 97 active+clean; 1.2 GiB data, 546 MiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 104 MiB/s wr, 111 op/s 2024-03-21T08:58:32.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:32 smithi100 ceph-mon[36306]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-03-21T08:58:34.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:34 smithi087 ceph-mon[32385]: pgmap v395: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 324 MiB/s wr, 258 op/s 2024-03-21T08:58:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:34 smithi105 ceph-mon[35993]: pgmap v395: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 324 MiB/s wr, 258 op/s 2024-03-21T08:58:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:34 smithi100 ceph-mon[36306]: pgmap v395: 97 pgs: 97 active+clean; 3.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 681 KiB/s rd, 324 MiB/s wr, 258 op/s 2024-03-21T08:58:36.422 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:36 smithi087 ceph-mon[32385]: pgmap v396: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 477 MiB/s wr, 343 op/s 2024-03-21T08:58:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:36 smithi105 ceph-mon[35993]: pgmap v396: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 477 MiB/s wr, 343 op/s 2024-03-21T08:58:36.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:36 smithi100 ceph-mon[36306]: pgmap v396: 97 pgs: 97 active+clean; 5.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 836 KiB/s rd, 477 MiB/s wr, 343 op/s 2024-03-21T08:58:38.412 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:38 smithi087 ceph-mon[32385]: pgmap v397: 97 pgs: 97 active+clean; 6.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 561 MiB/s wr, 365 op/s 2024-03-21T08:58:38.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:38 smithi105 ceph-mon[35993]: pgmap v397: 97 pgs: 97 active+clean; 6.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 561 MiB/s wr, 365 op/s 2024-03-21T08:58:38.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:38 smithi100 ceph-mon[36306]: pgmap v397: 97 pgs: 97 active+clean; 6.6 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 834 KiB/s rd, 561 MiB/s wr, 365 op/s 2024-03-21T08:58:40.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:40 smithi087 ceph-mon[32385]: pgmap v398: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 767 MiB/s wr, 560 op/s 2024-03-21T08:58:40.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:40 smithi105 ceph-mon[35993]: pgmap v398: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 767 MiB/s wr, 560 op/s 2024-03-21T08:58:40.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:40 smithi100 ceph-mon[36306]: pgmap v398: 97 pgs: 97 active+clean; 8.8 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1022 KiB/s rd, 767 MiB/s wr, 560 op/s 2024-03-21T08:58:42.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:42 smithi087 ceph-mon[32385]: pgmap v399: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 MiB/s wr, 772 op/s 2024-03-21T08:58:42.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:42 smithi105 ceph-mon[35993]: pgmap v399: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 MiB/s wr, 772 op/s 2024-03-21T08:58:42.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:42 smithi100 ceph-mon[36306]: pgmap v399: 97 pgs: 97 active+clean; 9.4 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 803 MiB/s wr, 772 op/s 2024-03-21T08:58:43.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:43 smithi105 ceph-mon[35993]: pgmap v400: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 966 MiB/s wr, 1.40k op/s 2024-03-21T08:58:43.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:43 smithi100 ceph-mon[36306]: pgmap v400: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 966 MiB/s wr, 1.40k op/s 2024-03-21T08:58:43.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:43 smithi087 ceph-mon[32385]: pgmap v400: 97 pgs: 97 active+clean; 9.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 966 MiB/s wr, 1.40k op/s 2024-03-21T08:58:46.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:45 smithi105 ceph-mon[35993]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 900 MiB/s wr, 1.63k op/s 2024-03-21T08:58:46.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:45 smithi100 ceph-mon[36306]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 900 MiB/s wr, 1.63k op/s 2024-03-21T08:58:46.416 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:45 smithi087 ceph-mon[32385]: pgmap v401: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 900 MiB/s wr, 1.63k op/s 2024-03-21T08:58:48.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:47 smithi100 ceph-mon[36306]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 MiB/s wr, 1.71k op/s 2024-03-21T08:58:48.418 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:47 smithi087 ceph-mon[32385]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 MiB/s wr, 1.71k op/s 2024-03-21T08:58:48.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:47 smithi105 ceph-mon[35993]: pgmap v402: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 835 MiB/s wr, 1.71k op/s 2024-03-21T08:58:50.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:50 smithi100 ceph-mon[36306]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 961 MiB/s wr, 2.17k op/s 2024-03-21T08:58:50.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:50 smithi087 ceph-mon[32385]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 961 MiB/s wr, 2.17k op/s 2024-03-21T08:58:50.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:50 smithi105 ceph-mon[35993]: pgmap v403: 97 pgs: 97 active+clean; 10 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 961 MiB/s wr, 2.17k op/s 2024-03-21T08:58:52.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:52 smithi100 ceph-mon[36306]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 841 MiB/s wr, 2.18k op/s 2024-03-21T08:58:52.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:52 smithi087 ceph-mon[32385]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 841 MiB/s wr, 2.18k op/s 2024-03-21T08:58:52.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:52 smithi105 ceph-mon[35993]: pgmap v404: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 841 MiB/s wr, 2.18k op/s 2024-03-21T08:58:54.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:54 smithi087 ceph-mon[32385]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 986 MiB/s wr, 2.45k op/s 2024-03-21T08:58:54.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:54 smithi105 ceph-mon[35993]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 986 MiB/s wr, 2.45k op/s 2024-03-21T08:58:54.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:54 smithi100 ceph-mon[36306]: pgmap v405: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 986 MiB/s wr, 2.45k op/s 2024-03-21T08:58:56.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:56 smithi087 ceph-mon[32385]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 923 MiB/s wr, 2.00k op/s 2024-03-21T08:58:56.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:56 smithi105 ceph-mon[35993]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 923 MiB/s wr, 2.00k op/s 2024-03-21T08:58:56.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:56 smithi100 ceph-mon[36306]: pgmap v406: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 923 MiB/s wr, 2.00k op/s 2024-03-21T08:58:58.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:58:58 smithi087 ceph-mon[32385]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 855 MiB/s wr, 1.85k op/s 2024-03-21T08:58:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:58:58 smithi105 ceph-mon[35993]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 855 MiB/s wr, 1.85k op/s 2024-03-21T08:58:58.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:58:58 smithi100 ceph-mon[36306]: pgmap v407: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 855 MiB/s wr, 1.85k op/s 2024-03-21T08:59:00.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:00 smithi087 ceph-mon[32385]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 974 MiB/s wr, 2.31k op/s 2024-03-21T08:59:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:00 smithi105 ceph-mon[35993]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 974 MiB/s wr, 2.31k op/s 2024-03-21T08:59:00.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:00 smithi100 ceph-mon[36306]: pgmap v408: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 974 MiB/s wr, 2.31k op/s 2024-03-21T08:59:02.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:02 smithi087 ceph-mon[32385]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 849 MiB/s wr, 2.04k op/s 2024-03-21T08:59:02.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:02 smithi105 ceph-mon[35993]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 849 MiB/s wr, 2.04k op/s 2024-03-21T08:59:02.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:02 smithi100 ceph-mon[36306]: pgmap v409: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 849 MiB/s wr, 2.04k op/s 2024-03-21T08:59:04.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:04 smithi087 ceph-mon[32385]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1007 MiB/s wr, 2.22k op/s 2024-03-21T08:59:04.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:04 smithi105 ceph-mon[35993]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1007 MiB/s wr, 2.22k op/s 2024-03-21T08:59:04.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:04 smithi100 ceph-mon[36306]: pgmap v410: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1007 MiB/s wr, 2.22k op/s 2024-03-21T08:59:06.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:06 smithi087 ceph-mon[32385]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 907 MiB/s wr, 1.96k op/s 2024-03-21T08:59:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:06 smithi105 ceph-mon[35993]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 907 MiB/s wr, 1.96k op/s 2024-03-21T08:59:06.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:06 smithi100 ceph-mon[36306]: pgmap v411: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 907 MiB/s wr, 1.96k op/s 2024-03-21T08:59:08.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:08 smithi087 ceph-mon[32385]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.87k op/s 2024-03-21T08:59:08.486 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:08 smithi105 ceph-mon[35993]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.87k op/s 2024-03-21T08:59:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:08 smithi100 ceph-mon[36306]: pgmap v412: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 833 MiB/s wr, 1.87k op/s 2024-03-21T08:59:10.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:10 smithi087 ceph-mon[32385]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 963 MiB/s wr, 2.31k op/s 2024-03-21T08:59:10.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:10 smithi105 ceph-mon[35993]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 963 MiB/s wr, 2.31k op/s 2024-03-21T08:59:10.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:10 smithi100 ceph-mon[36306]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 963 MiB/s wr, 2.31k op/s 2024-03-21T08:59:11.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:11 smithi105 ceph-mon[35993]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 836 MiB/s wr, 1.91k op/s 2024-03-21T08:59:11.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:11 smithi100 ceph-mon[36306]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 836 MiB/s wr, 1.91k op/s 2024-03-21T08:59:11.662 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:11 smithi087 ceph-mon[32385]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 836 MiB/s wr, 1.91k op/s 2024-03-21T08:59:14.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:13 smithi100 ceph-mon[36306]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 998 MiB/s wr, 2.18k op/s 2024-03-21T08:59:14.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:13 smithi087 ceph-mon[32385]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 998 MiB/s wr, 2.18k op/s 2024-03-21T08:59:14.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:13 smithi105 ceph-mon[35993]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 998 MiB/s wr, 2.18k op/s 2024-03-21T08:59:16.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:15 smithi100 ceph-mon[36306]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 880 MiB/s wr, 2.01k op/s 2024-03-21T08:59:16.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:15 smithi087 ceph-mon[32385]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 880 MiB/s wr, 2.01k op/s 2024-03-21T08:59:16.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:15 smithi105 ceph-mon[35993]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 880 MiB/s wr, 2.01k op/s 2024-03-21T08:59:18.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:18 smithi100 ceph-mon[36306]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 855 MiB/s wr, 1.99k op/s 2024-03-21T08:59:18.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:18 smithi087 ceph-mon[32385]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 855 MiB/s wr, 1.99k op/s 2024-03-21T08:59:18.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:18 smithi105 ceph-mon[35993]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 855 MiB/s wr, 1.99k op/s 2024-03-21T08:59:20.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:20 smithi087 ceph-mon[32385]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 960 MiB/s wr, 2.17k op/s 2024-03-21T08:59:20.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:20 smithi105 ceph-mon[35993]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 960 MiB/s wr, 2.17k op/s 2024-03-21T08:59:20.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:20 smithi100 ceph-mon[36306]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 960 MiB/s wr, 2.17k op/s 2024-03-21T08:59:22.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:22 smithi087 ceph-mon[32385]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 826 MiB/s wr, 1.66k op/s 2024-03-21T08:59:22.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:22 smithi105 ceph-mon[35993]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 826 MiB/s wr, 1.66k op/s 2024-03-21T08:59:22.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:22 smithi100 ceph-mon[36306]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 826 MiB/s wr, 1.66k op/s 2024-03-21T08:59:24.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:24 smithi087 ceph-mon[32385]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 2.30k op/s 2024-03-21T08:59:24.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:24 smithi105 ceph-mon[35993]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 2.30k op/s 2024-03-21T08:59:24.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:24 smithi100 ceph-mon[36306]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.0 GiB/s wr, 2.30k op/s 2024-03-21T08:59:26.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:26 smithi087 ceph-mon[32385]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 876 MiB/s wr, 2.27k op/s 2024-03-21T08:59:26.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:26 smithi105 ceph-mon[35993]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 876 MiB/s wr, 2.27k op/s 2024-03-21T08:59:26.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:26 smithi100 ceph-mon[36306]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 876 MiB/s wr, 2.27k op/s 2024-03-21T08:59:28.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:28 smithi087 ceph-mon[32385]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 2.50k op/s 2024-03-21T08:59:28.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:28 smithi105 ceph-mon[35993]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 2.50k op/s 2024-03-21T08:59:28.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:28 smithi100 ceph-mon[36306]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 852 MiB/s wr, 2.50k op/s 2024-03-21T08:59:30.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:30 smithi087 ceph-mon[32385]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 921 MiB/s wr, 2.84k op/s 2024-03-21T08:59:30.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:30 smithi105 ceph-mon[35993]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 921 MiB/s wr, 2.84k op/s 2024-03-21T08:59:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:30 smithi100 ceph-mon[36306]: pgmap v423: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 921 MiB/s wr, 2.84k op/s 2024-03-21T08:59:31.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:31 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:59:31.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:31 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:59:31.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:31 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T08:59:32.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:32 smithi087 ceph-mon[32385]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 821 MiB/s wr, 2.58k op/s 2024-03-21T08:59:32.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:32 smithi105 ceph-mon[35993]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 821 MiB/s wr, 2.58k op/s 2024-03-21T08:59:32.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:32 smithi100 ceph-mon[36306]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 821 MiB/s wr, 2.58k op/s 2024-03-21T08:59:33.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:33.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:33 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:33.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:33.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:33 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:33.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:33.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:33 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:34 smithi105 ceph-mon[35993]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 991 MiB/s wr, 3.00k op/s 2024-03-21T08:59:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:34 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:59:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:34 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:59:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:34 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:34 smithi100 ceph-mon[36306]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 991 MiB/s wr, 3.00k op/s 2024-03-21T08:59:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:34 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:59:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:34 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:59:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:34 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:34.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:34 smithi087 ceph-mon[32385]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 991 MiB/s wr, 3.00k op/s 2024-03-21T08:59:34.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:34 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T08:59:34.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:34 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T08:59:34.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:34 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T08:59:35.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:35 smithi105 ceph-mon[35993]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 2.39k op/s 2024-03-21T08:59:35.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:35 smithi100 ceph-mon[36306]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 2.39k op/s 2024-03-21T08:59:35.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:35 smithi087 ceph-mon[32385]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 818 MiB/s wr, 2.39k op/s 2024-03-21T08:59:38.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:38 smithi087 ceph-mon[32385]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 821 MiB/s wr, 2.07k op/s 2024-03-21T08:59:38.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:38 smithi105 ceph-mon[35993]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 821 MiB/s wr, 2.07k op/s 2024-03-21T08:59:38.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:38 smithi100 ceph-mon[36306]: pgmap v427: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 821 MiB/s wr, 2.07k op/s 2024-03-21T08:59:40.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:40 smithi087 ceph-mon[32385]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 918 MiB/s wr, 2.14k op/s 2024-03-21T08:59:40.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:40 smithi105 ceph-mon[35993]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 918 MiB/s wr, 2.14k op/s 2024-03-21T08:59:40.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:40 smithi100 ceph-mon[36306]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 918 MiB/s wr, 2.14k op/s 2024-03-21T08:59:42.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:42 smithi087 ceph-mon[32385]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 829 MiB/s wr, 1.82k op/s 2024-03-21T08:59:42.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:42 smithi105 ceph-mon[35993]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 829 MiB/s wr, 1.82k op/s 2024-03-21T08:59:42.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:42 smithi100 ceph-mon[36306]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 829 MiB/s wr, 1.82k op/s 2024-03-21T08:59:44.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:44 smithi087 ceph-mon[32385]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.18k op/s 2024-03-21T08:59:44.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:44 smithi105 ceph-mon[35993]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.18k op/s 2024-03-21T08:59:44.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:44 smithi100 ceph-mon[36306]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.18k op/s 2024-03-21T08:59:46.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:46 smithi087 ceph-mon[32385]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.85k op/s 2024-03-21T08:59:46.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:46 smithi105 ceph-mon[35993]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.85k op/s 2024-03-21T08:59:46.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:46 smithi100 ceph-mon[36306]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 876 MiB/s wr, 1.85k op/s 2024-03-21T08:59:48.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:48 smithi087 ceph-mon[32385]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 833 MiB/s wr, 1.83k op/s 2024-03-21T08:59:48.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:48 smithi105 ceph-mon[35993]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 833 MiB/s wr, 1.83k op/s 2024-03-21T08:59:48.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:48 smithi100 ceph-mon[36306]: pgmap v432: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 833 MiB/s wr, 1.83k op/s 2024-03-21T08:59:50.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:50 smithi087 ceph-mon[32385]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 944 MiB/s wr, 2.27k op/s 2024-03-21T08:59:50.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:50 smithi105 ceph-mon[35993]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 944 MiB/s wr, 2.27k op/s 2024-03-21T08:59:50.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:50 smithi100 ceph-mon[36306]: pgmap v433: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 944 MiB/s wr, 2.27k op/s 2024-03-21T08:59:52.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:52 smithi087 ceph-mon[32385]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 829 MiB/s wr, 2.02k op/s 2024-03-21T08:59:52.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:52 smithi105 ceph-mon[35993]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 829 MiB/s wr, 2.02k op/s 2024-03-21T08:59:52.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:52 smithi100 ceph-mon[36306]: pgmap v434: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 829 MiB/s wr, 2.02k op/s 2024-03-21T08:59:54.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:54 smithi105 ceph-mon[35993]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.60k op/s 2024-03-21T08:59:54.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:54 smithi100 ceph-mon[36306]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.60k op/s 2024-03-21T08:59:54.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:54 smithi087 ceph-mon[32385]: pgmap v435: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.60k op/s 2024-03-21T08:59:56.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:56 smithi105 ceph-mon[35993]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 855 MiB/s wr, 2.46k op/s 2024-03-21T08:59:56.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:56 smithi100 ceph-mon[36306]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 855 MiB/s wr, 2.46k op/s 2024-03-21T08:59:56.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:56 smithi087 ceph-mon[32385]: pgmap v436: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 855 MiB/s wr, 2.46k op/s 2024-03-21T08:59:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 08:59:58 smithi105 ceph-mon[35993]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 MiB/s wr, 2.54k op/s 2024-03-21T08:59:58.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 08:59:58 smithi100 ceph-mon[36306]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 MiB/s wr, 2.54k op/s 2024-03-21T08:59:58.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 08:59:58 smithi087 ceph-mon[32385]: pgmap v437: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 MiB/s wr, 2.54k op/s 2024-03-21T09:00:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:00 smithi105 ceph-mon[35993]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 926 MiB/s wr, 2.89k op/s 2024-03-21T09:00:00.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:00 smithi105 ceph-mon[35993]: overall HEALTH_OK 2024-03-21T09:00:00.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:00 smithi100 ceph-mon[36306]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 926 MiB/s wr, 2.89k op/s 2024-03-21T09:00:00.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:00 smithi100 ceph-mon[36306]: overall HEALTH_OK 2024-03-21T09:00:00.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:00 smithi087 ceph-mon[32385]: pgmap v438: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 926 MiB/s wr, 2.89k op/s 2024-03-21T09:00:00.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:00 smithi087 ceph-mon[32385]: overall HEALTH_OK 2024-03-21T09:00:02.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:02 smithi105 ceph-mon[35993]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 821 MiB/s wr, 2.66k op/s 2024-03-21T09:00:02.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:02 smithi100 ceph-mon[36306]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 821 MiB/s wr, 2.66k op/s 2024-03-21T09:00:02.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:02 smithi087 ceph-mon[32385]: pgmap v439: 97 pgs: 97 active+clean; 10 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 821 MiB/s wr, 2.66k op/s 2024-03-21T09:00:03.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:03 smithi087 ceph-mon[32385]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 989 MiB/s wr, 2.94k op/s 2024-03-21T09:00:03.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:03 smithi105 ceph-mon[35993]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 989 MiB/s wr, 2.94k op/s 2024-03-21T09:00:03.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:03 smithi100 ceph-mon[36306]: pgmap v440: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 989 MiB/s wr, 2.94k op/s 2024-03-21T09:00:06.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:06 smithi087 ceph-mon[32385]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 841 MiB/s wr, 2.29k op/s 2024-03-21T09:00:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:06 smithi105 ceph-mon[35993]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 841 MiB/s wr, 2.29k op/s 2024-03-21T09:00:06.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:06 smithi100 ceph-mon[36306]: pgmap v441: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 841 MiB/s wr, 2.29k op/s 2024-03-21T09:00:08.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:08 smithi087 ceph-mon[32385]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 2.13k op/s 2024-03-21T09:00:08.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:08 smithi105 ceph-mon[35993]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 2.13k op/s 2024-03-21T09:00:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:08 smithi100 ceph-mon[36306]: pgmap v442: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 826 MiB/s wr, 2.13k op/s 2024-03-21T09:00:10.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:10 smithi087 ceph-mon[32385]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 2.31k op/s 2024-03-21T09:00:10.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:10 smithi105 ceph-mon[35993]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 2.31k op/s 2024-03-21T09:00:10.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:10 smithi100 ceph-mon[36306]: pgmap v443: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 2.31k op/s 2024-03-21T09:00:12.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:12 smithi087 ceph-mon[32385]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 823 MiB/s wr, 2.00k op/s 2024-03-21T09:00:12.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:12 smithi105 ceph-mon[35993]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 823 MiB/s wr, 2.00k op/s 2024-03-21T09:00:12.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:12 smithi100 ceph-mon[36306]: pgmap v444: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 823 MiB/s wr, 2.00k op/s 2024-03-21T09:00:14.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:14 smithi087 ceph-mon[32385]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 978 MiB/s wr, 2.14k op/s 2024-03-21T09:00:14.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:14 smithi105 ceph-mon[35993]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 978 MiB/s wr, 2.14k op/s 2024-03-21T09:00:14.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:14 smithi100 ceph-mon[36306]: pgmap v445: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 978 MiB/s wr, 2.14k op/s 2024-03-21T09:00:16.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:16 smithi087 ceph-mon[32385]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 838 MiB/s wr, 1.75k op/s 2024-03-21T09:00:16.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:16 smithi105 ceph-mon[35993]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 838 MiB/s wr, 1.75k op/s 2024-03-21T09:00:16.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:16 smithi100 ceph-mon[36306]: pgmap v446: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 838 MiB/s wr, 1.75k op/s 2024-03-21T09:00:18.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:18 smithi087 ceph-mon[32385]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 825 MiB/s wr, 1.85k op/s 2024-03-21T09:00:18.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:18 smithi105 ceph-mon[35993]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 825 MiB/s wr, 1.85k op/s 2024-03-21T09:00:18.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:18 smithi100 ceph-mon[36306]: pgmap v447: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 825 MiB/s wr, 1.85k op/s 2024-03-21T09:00:20.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:20 smithi087 ceph-mon[32385]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 924 MiB/s wr, 2.42k op/s 2024-03-21T09:00:20.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:20 smithi105 ceph-mon[35993]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 924 MiB/s wr, 2.42k op/s 2024-03-21T09:00:20.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:20 smithi100 ceph-mon[36306]: pgmap v448: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 924 MiB/s wr, 2.42k op/s 2024-03-21T09:00:22.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:22 smithi087 ceph-mon[32385]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 822 MiB/s wr, 2.32k op/s 2024-03-21T09:00:22.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:22 smithi105 ceph-mon[35993]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 822 MiB/s wr, 2.32k op/s 2024-03-21T09:00:22.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:22 smithi100 ceph-mon[36306]: pgmap v449: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 822 MiB/s wr, 2.32k op/s 2024-03-21T09:00:24.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:24 smithi087 ceph-mon[32385]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 980 MiB/s wr, 2.88k op/s 2024-03-21T09:00:24.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:24 smithi105 ceph-mon[35993]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 980 MiB/s wr, 2.88k op/s 2024-03-21T09:00:24.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:24 smithi100 ceph-mon[36306]: pgmap v450: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 980 MiB/s wr, 2.88k op/s 2024-03-21T09:00:26.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:26 smithi105 ceph-mon[35993]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 857 MiB/s wr, 2.60k op/s 2024-03-21T09:00:26.517 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:26 smithi100 ceph-mon[36306]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 857 MiB/s wr, 2.60k op/s 2024-03-21T09:00:26.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:26 smithi087 ceph-mon[32385]: pgmap v451: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 857 MiB/s wr, 2.60k op/s 2024-03-21T09:00:28.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:28 smithi105 ceph-mon[35993]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 839 MiB/s wr, 2.66k op/s 2024-03-21T09:00:28.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:28 smithi100 ceph-mon[36306]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 839 MiB/s wr, 2.66k op/s 2024-03-21T09:00:28.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:28 smithi087 ceph-mon[32385]: pgmap v452: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 839 MiB/s wr, 2.66k op/s 2024-03-21T09:00:30.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:30 smithi105 ceph-mon[35993]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 929 MiB/s wr, 2.88k op/s 2024-03-21T09:00:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:30 smithi100 ceph-mon[36306]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 929 MiB/s wr, 2.88k op/s 2024-03-21T09:00:30.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:30 smithi087 ceph-mon[32385]: pgmap v453: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 929 MiB/s wr, 2.88k op/s 2024-03-21T09:00:32.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:32 smithi105 ceph-mon[35993]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 820 MiB/s wr, 2.27k op/s 2024-03-21T09:00:32.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:32 smithi100 ceph-mon[36306]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 820 MiB/s wr, 2.27k op/s 2024-03-21T09:00:32.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:32 smithi087 ceph-mon[32385]: pgmap v454: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 820 MiB/s wr, 2.27k op/s 2024-03-21T09:00:34.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:34 smithi105 ceph-mon[35993]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 994 MiB/s wr, 2.26k op/s 2024-03-21T09:00:34.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:34 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:34 smithi100 ceph-mon[36306]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 994 MiB/s wr, 2.26k op/s 2024-03-21T09:00:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:34 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:34.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:34 smithi087 ceph-mon[32385]: pgmap v455: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 994 MiB/s wr, 2.26k op/s 2024-03-21T09:00:34.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:34 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:00:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:36 smithi105 ceph-mon[35993]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 872 MiB/s wr, 1.65k op/s 2024-03-21T09:00:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:36 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:00:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:36 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:00:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:36 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:00:36.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:36 smithi100 ceph-mon[36306]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 872 MiB/s wr, 1.65k op/s 2024-03-21T09:00:36.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:36 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:00:36.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:36 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:00:36.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:36 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:00:36.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:36 smithi087 ceph-mon[32385]: pgmap v456: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 872 MiB/s wr, 1.65k op/s 2024-03-21T09:00:36.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:00:36.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:00:36.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:36 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:00:38.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:38 smithi100 ceph-mon[36306]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 851 MiB/s wr, 1.60k op/s 2024-03-21T09:00:38.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:38 smithi087 ceph-mon[32385]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 851 MiB/s wr, 1.60k op/s 2024-03-21T09:00:38.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:38 smithi105 ceph-mon[35993]: pgmap v457: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 851 MiB/s wr, 1.60k op/s 2024-03-21T09:00:40.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:40 smithi087 ceph-mon[32385]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 949 MiB/s wr, 1.80k op/s 2024-03-21T09:00:40.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:40 smithi105 ceph-mon[35993]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 949 MiB/s wr, 1.80k op/s 2024-03-21T09:00:40.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:40 smithi100 ceph-mon[36306]: pgmap v458: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 949 MiB/s wr, 1.80k op/s 2024-03-21T09:00:42.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:42 smithi087 ceph-mon[32385]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 MiB/s wr, 1.53k op/s 2024-03-21T09:00:42.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:42 smithi105 ceph-mon[35993]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 MiB/s wr, 1.53k op/s 2024-03-21T09:00:42.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:42 smithi100 ceph-mon[36306]: pgmap v459: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 834 MiB/s wr, 1.53k op/s 2024-03-21T09:00:44.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:44 smithi087 ceph-mon[32385]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1007 MiB/s wr, 2.10k op/s 2024-03-21T09:00:44.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:44 smithi105 ceph-mon[35993]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1007 MiB/s wr, 2.10k op/s 2024-03-21T09:00:44.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:44 smithi100 ceph-mon[36306]: pgmap v460: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1007 MiB/s wr, 2.10k op/s 2024-03-21T09:00:46.568 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:46 smithi100 ceph-mon[36306]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 879 MiB/s wr, 1.98k op/s 2024-03-21T09:00:46.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:46 smithi087 ceph-mon[32385]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 879 MiB/s wr, 1.98k op/s 2024-03-21T09:00:46.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:46 smithi105 ceph-mon[35993]: pgmap v461: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 879 MiB/s wr, 1.98k op/s 2024-03-21T09:00:47.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:47 smithi087 ceph-mon[32385]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 853 MiB/s wr, 2.01k op/s 2024-03-21T09:00:47.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:47 smithi105 ceph-mon[35993]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 853 MiB/s wr, 2.01k op/s 2024-03-21T09:00:47.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:47 smithi100 ceph-mon[36306]: pgmap v462: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 922 KiB/s rd, 853 MiB/s wr, 2.01k op/s 2024-03-21T09:00:50.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:50 smithi087 ceph-mon[32385]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 939 MiB/s wr, 2.19k op/s 2024-03-21T09:00:50.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:50 smithi105 ceph-mon[35993]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 939 MiB/s wr, 2.19k op/s 2024-03-21T09:00:50.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:50 smithi100 ceph-mon[36306]: pgmap v463: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 939 MiB/s wr, 2.19k op/s 2024-03-21T09:00:52.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:52 smithi087 ceph-mon[32385]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 822 MiB/s wr, 1.89k op/s 2024-03-21T09:00:52.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:52 smithi105 ceph-mon[35993]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 822 MiB/s wr, 1.89k op/s 2024-03-21T09:00:52.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:52 smithi100 ceph-mon[36306]: pgmap v464: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 822 MiB/s wr, 1.89k op/s 2024-03-21T09:00:54.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:54 smithi087 ceph-mon[32385]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.15k op/s 2024-03-21T09:00:54.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:54 smithi105 ceph-mon[35993]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.15k op/s 2024-03-21T09:00:54.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:54 smithi100 ceph-mon[36306]: pgmap v465: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1006 MiB/s wr, 2.15k op/s 2024-03-21T09:00:56.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:56 smithi087 ceph-mon[32385]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 872 MiB/s wr, 1.70k op/s 2024-03-21T09:00:56.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:56 smithi105 ceph-mon[35993]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 872 MiB/s wr, 1.70k op/s 2024-03-21T09:00:56.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:56 smithi100 ceph-mon[36306]: pgmap v466: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 872 MiB/s wr, 1.70k op/s 2024-03-21T09:00:58.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:00:58 smithi105 ceph-mon[35993]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 799 MiB/s wr, 1.50k op/s 2024-03-21T09:00:58.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:00:58 smithi100 ceph-mon[36306]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 799 MiB/s wr, 1.50k op/s 2024-03-21T09:00:58.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:00:58 smithi087 ceph-mon[32385]: pgmap v467: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 799 MiB/s wr, 1.50k op/s 2024-03-21T09:01:00.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:00 smithi105 ceph-mon[35993]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 1.63k op/s 2024-03-21T09:01:00.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:00 smithi100 ceph-mon[36306]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 1.63k op/s 2024-03-21T09:01:00.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:00 smithi087 ceph-mon[32385]: pgmap v468: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 932 MiB/s wr, 1.63k op/s 2024-03-21T09:01:02.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:02 smithi105 ceph-mon[35993]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 827 MiB/s wr, 1.43k op/s 2024-03-21T09:01:02.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:02 smithi100 ceph-mon[36306]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 827 MiB/s wr, 1.43k op/s 2024-03-21T09:01:02.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:02 smithi087 ceph-mon[32385]: pgmap v469: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 827 MiB/s wr, 1.43k op/s 2024-03-21T09:01:04.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:04 smithi105 ceph-mon[35993]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1008 MiB/s wr, 2.10k op/s 2024-03-21T09:01:04.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:04 smithi100 ceph-mon[36306]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1008 MiB/s wr, 2.10k op/s 2024-03-21T09:01:04.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:04 smithi087 ceph-mon[32385]: pgmap v470: 97 pgs: 97 active+clean; 10 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1008 MiB/s wr, 2.10k op/s 2024-03-21T09:01:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:06 smithi105 ceph-mon[35993]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 864 MiB/s wr, 1.99k op/s 2024-03-21T09:01:06.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:06 smithi100 ceph-mon[36306]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 864 MiB/s wr, 1.99k op/s 2024-03-21T09:01:06.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:06 smithi087 ceph-mon[32385]: pgmap v471: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 864 MiB/s wr, 1.99k op/s 2024-03-21T09:01:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:08 smithi100 ceph-mon[36306]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 1.81k op/s 2024-03-21T09:01:08.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:08 smithi087 ceph-mon[32385]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 1.81k op/s 2024-03-21T09:01:08.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:08 smithi105 ceph-mon[35993]: pgmap v472: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 1.81k op/s 2024-03-21T09:01:10.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:10 smithi087 ceph-mon[32385]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 961 MiB/s wr, 2.03k op/s 2024-03-21T09:01:10.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:10 smithi105 ceph-mon[35993]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 961 MiB/s wr, 2.03k op/s 2024-03-21T09:01:10.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:10 smithi100 ceph-mon[36306]: pgmap v473: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 961 MiB/s wr, 2.03k op/s 2024-03-21T09:01:12.659 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:12 smithi087 ceph-mon[32385]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 1.83k op/s 2024-03-21T09:01:12.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:12 smithi105 ceph-mon[35993]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 1.83k op/s 2024-03-21T09:01:12.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:12 smithi100 ceph-mon[36306]: pgmap v474: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 819 MiB/s wr, 1.83k op/s 2024-03-21T09:01:14.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:14 smithi087 ceph-mon[32385]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1003 MiB/s wr, 2.31k op/s 2024-03-21T09:01:14.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:14 smithi105 ceph-mon[35993]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1003 MiB/s wr, 2.31k op/s 2024-03-21T09:01:14.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:14 smithi100 ceph-mon[36306]: pgmap v475: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 1003 MiB/s wr, 2.31k op/s 2024-03-21T09:01:16.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:16 smithi087 ceph-mon[32385]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 865 MiB/s wr, 1.70k op/s 2024-03-21T09:01:16.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:16 smithi105 ceph-mon[35993]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 865 MiB/s wr, 1.70k op/s 2024-03-21T09:01:16.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:16 smithi100 ceph-mon[36306]: pgmap v476: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 865 MiB/s wr, 1.70k op/s 2024-03-21T09:01:18.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:18 smithi087 ceph-mon[32385]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 805 MiB/s wr, 1.43k op/s 2024-03-21T09:01:18.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:18 smithi105 ceph-mon[35993]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 805 MiB/s wr, 1.43k op/s 2024-03-21T09:01:18.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:18 smithi100 ceph-mon[36306]: pgmap v477: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 805 MiB/s wr, 1.43k op/s 2024-03-21T09:01:20.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:20 smithi087 ceph-mon[32385]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 983 MiB/s wr, 1.77k op/s 2024-03-21T09:01:20.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:20 smithi105 ceph-mon[35993]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 983 MiB/s wr, 1.77k op/s 2024-03-21T09:01:20.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:20 smithi100 ceph-mon[36306]: pgmap v478: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 983 MiB/s wr, 1.77k op/s 2024-03-21T09:01:22.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:22 smithi087 ceph-mon[32385]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 MiB/s wr, 1.63k op/s 2024-03-21T09:01:22.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:22 smithi105 ceph-mon[35993]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 MiB/s wr, 1.63k op/s 2024-03-21T09:01:22.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:22 smithi100 ceph-mon[36306]: pgmap v479: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 843 MiB/s wr, 1.63k op/s 2024-03-21T09:01:24.613 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:24 smithi087 ceph-mon[32385]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1016 MiB/s wr, 1.90k op/s 2024-03-21T09:01:24.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:24 smithi105 ceph-mon[35993]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1016 MiB/s wr, 1.90k op/s 2024-03-21T09:01:24.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:24 smithi100 ceph-mon[36306]: pgmap v480: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1016 MiB/s wr, 1.90k op/s 2024-03-21T09:01:26.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:26 smithi105 ceph-mon[35993]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 863 MiB/s wr, 1.57k op/s 2024-03-21T09:01:26.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:26 smithi100 ceph-mon[36306]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 863 MiB/s wr, 1.57k op/s 2024-03-21T09:01:26.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:26 smithi087 ceph-mon[32385]: pgmap v481: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 863 MiB/s wr, 1.57k op/s 2024-03-21T09:01:27.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:27 smithi105 ceph-mon[35993]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 791 MiB/s wr, 1.56k op/s 2024-03-21T09:01:27.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:27 smithi100 ceph-mon[36306]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 791 MiB/s wr, 1.56k op/s 2024-03-21T09:01:27.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:27 smithi087 ceph-mon[32385]: pgmap v482: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 791 MiB/s wr, 1.56k op/s 2024-03-21T09:01:30.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:30 smithi087 ceph-mon[32385]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 948 MiB/s wr, 2.17k op/s 2024-03-21T09:01:30.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:30 smithi105 ceph-mon[35993]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 948 MiB/s wr, 2.17k op/s 2024-03-21T09:01:30.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:30 smithi100 ceph-mon[36306]: pgmap v483: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 948 MiB/s wr, 2.17k op/s 2024-03-21T09:01:32.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:32 smithi087 ceph-mon[32385]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 MiB/s wr, 1.90k op/s 2024-03-21T09:01:32.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:32 smithi105 ceph-mon[35993]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 MiB/s wr, 1.90k op/s 2024-03-21T09:01:32.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:32 smithi100 ceph-mon[36306]: pgmap v484: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 800 MiB/s wr, 1.90k op/s 2024-03-21T09:01:34.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:34 smithi105 ceph-mon[35993]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 967 MiB/s wr, 2.05k op/s 2024-03-21T09:01:34.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:34 smithi100 ceph-mon[36306]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 967 MiB/s wr, 2.05k op/s 2024-03-21T09:01:34.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:34 smithi087 ceph-mon[32385]: pgmap v485: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 967 MiB/s wr, 2.05k op/s 2024-03-21T09:01:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:36 smithi105 ceph-mon[35993]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 820 MiB/s wr, 1.85k op/s 2024-03-21T09:01:36.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:36 smithi100 ceph-mon[36306]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 820 MiB/s wr, 1.85k op/s 2024-03-21T09:01:36.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:36 smithi087 ceph-mon[32385]: pgmap v486: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 820 MiB/s wr, 1.85k op/s 2024-03-21T09:01:37.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:37 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:37.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:37 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:37.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:37 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:01:38.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:38 smithi100 ceph-mon[36306]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 769 MiB/s wr, 1.71k op/s 2024-03-21T09:01:38.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:38 smithi087 ceph-mon[32385]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 769 MiB/s wr, 1.71k op/s 2024-03-21T09:01:38.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:38 smithi105 ceph-mon[35993]: pgmap v487: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 769 MiB/s wr, 1.71k op/s 2024-03-21T09:01:39.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:39.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:39.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:39 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:01:39.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:39 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:39.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:39 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:39.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:39 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:01:39.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:39 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:01:39.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:39 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:01:39.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:39 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:01:40.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:40 smithi087 ceph-mon[32385]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 953 MiB/s wr, 2.10k op/s 2024-03-21T09:01:40.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:40 smithi105 ceph-mon[35993]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 953 MiB/s wr, 2.10k op/s 2024-03-21T09:01:40.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:40 smithi100 ceph-mon[36306]: pgmap v488: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 953 MiB/s wr, 2.10k op/s 2024-03-21T09:01:42.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:42 smithi087 ceph-mon[32385]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 827 MiB/s wr, 1.50k op/s 2024-03-21T09:01:42.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:42 smithi105 ceph-mon[35993]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 827 MiB/s wr, 1.50k op/s 2024-03-21T09:01:42.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:42 smithi100 ceph-mon[36306]: pgmap v489: 97 pgs: 97 active+clean; 10 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 827 MiB/s wr, 1.50k op/s 2024-03-21T09:01:44.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:44 smithi087 ceph-mon[32385]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 991 MiB/s wr, 1.78k op/s 2024-03-21T09:01:44.734 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:44 smithi105 ceph-mon[35993]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 991 MiB/s wr, 1.78k op/s 2024-03-21T09:01:44.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:44 smithi100 ceph-mon[36306]: pgmap v490: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 991 MiB/s wr, 1.78k op/s 2024-03-21T09:01:46.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:46 smithi087 ceph-mon[32385]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 814 MiB/s wr, 1.61k op/s 2024-03-21T09:01:46.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:46 smithi105 ceph-mon[35993]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 814 MiB/s wr, 1.61k op/s 2024-03-21T09:01:46.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:46 smithi100 ceph-mon[36306]: pgmap v491: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 814 MiB/s wr, 1.61k op/s 2024-03-21T09:01:48.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:48 smithi087 ceph-mon[32385]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 803 MiB/s wr, 1.56k op/s 2024-03-21T09:01:48.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:48 smithi105 ceph-mon[35993]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 803 MiB/s wr, 1.56k op/s 2024-03-21T09:01:48.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:48 smithi100 ceph-mon[36306]: pgmap v492: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 803 MiB/s wr, 1.56k op/s 2024-03-21T09:01:50.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:50 smithi087 ceph-mon[32385]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 977 MiB/s wr, 1.81k op/s 2024-03-21T09:01:50.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:50 smithi105 ceph-mon[35993]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 977 MiB/s wr, 1.81k op/s 2024-03-21T09:01:50.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:50 smithi100 ceph-mon[36306]: pgmap v493: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 977 MiB/s wr, 1.81k op/s 2024-03-21T09:01:52.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:52 smithi087 ceph-mon[32385]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 MiB/s wr, 1.45k op/s 2024-03-21T09:01:52.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:52 smithi105 ceph-mon[35993]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 MiB/s wr, 1.45k op/s 2024-03-21T09:01:52.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:52 smithi100 ceph-mon[36306]: pgmap v494: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 827 MiB/s wr, 1.45k op/s 2024-03-21T09:01:54.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:54 smithi087 ceph-mon[32385]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 982 MiB/s wr, 2.08k op/s 2024-03-21T09:01:54.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:54 smithi105 ceph-mon[35993]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 982 MiB/s wr, 2.08k op/s 2024-03-21T09:01:54.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:54 smithi100 ceph-mon[36306]: pgmap v495: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 982 MiB/s wr, 2.08k op/s 2024-03-21T09:01:56.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:56 smithi105 ceph-mon[35993]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.86k op/s 2024-03-21T09:01:56.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:56 smithi100 ceph-mon[36306]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.86k op/s 2024-03-21T09:01:56.917 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:56 smithi087 ceph-mon[32385]: pgmap v496: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 815 MiB/s wr, 1.86k op/s 2024-03-21T09:01:58.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:01:58 smithi105 ceph-mon[35993]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.85k op/s 2024-03-21T09:01:58.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:01:58 smithi100 ceph-mon[36306]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.85k op/s 2024-03-21T09:01:58.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:01:58 smithi087 ceph-mon[32385]: pgmap v497: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 847 MiB/s wr, 1.85k op/s 2024-03-21T09:02:00.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:00 smithi105 ceph-mon[35993]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 954 MiB/s wr, 1.97k op/s 2024-03-21T09:02:00.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:00 smithi100 ceph-mon[36306]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 954 MiB/s wr, 1.97k op/s 2024-03-21T09:02:00.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:00 smithi087 ceph-mon[32385]: pgmap v498: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 954 MiB/s wr, 1.97k op/s 2024-03-21T09:02:02.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:02 smithi105 ceph-mon[35993]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 831 MiB/s wr, 1.82k op/s 2024-03-21T09:02:02.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:02 smithi100 ceph-mon[36306]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 831 MiB/s wr, 1.82k op/s 2024-03-21T09:02:02.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:02 smithi087 ceph-mon[32385]: pgmap v499: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 831 MiB/s wr, 1.82k op/s 2024-03-21T09:02:04.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:04 smithi105 ceph-mon[35993]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1000 MiB/s wr, 2.17k op/s 2024-03-21T09:02:04.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:04 smithi100 ceph-mon[36306]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1000 MiB/s wr, 2.17k op/s 2024-03-21T09:02:04.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:04 smithi087 ceph-mon[32385]: pgmap v500: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1000 MiB/s wr, 2.17k op/s 2024-03-21T09:02:05.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:05 smithi087 ceph-mon[32385]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 840 MiB/s wr, 1.60k op/s 2024-03-21T09:02:05.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:05 smithi105 ceph-mon[35993]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 840 MiB/s wr, 1.60k op/s 2024-03-21T09:02:06.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:05 smithi100 ceph-mon[36306]: pgmap v501: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 840 MiB/s wr, 1.60k op/s 2024-03-21T09:02:08.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:08 smithi105 ceph-mon[35993]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 862 MiB/s wr, 1.72k op/s 2024-03-21T09:02:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:08 smithi100 ceph-mon[36306]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 862 MiB/s wr, 1.72k op/s 2024-03-21T09:02:08.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:08 smithi087 ceph-mon[32385]: pgmap v502: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 862 MiB/s wr, 1.72k op/s 2024-03-21T09:02:10.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:10 smithi105 ceph-mon[35993]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 952 MiB/s wr, 2.14k op/s 2024-03-21T09:02:10.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:10 smithi100 ceph-mon[36306]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 952 MiB/s wr, 2.14k op/s 2024-03-21T09:02:10.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:10 smithi087 ceph-mon[32385]: pgmap v503: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 952 MiB/s wr, 2.14k op/s 2024-03-21T09:02:12.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:12 smithi105 ceph-mon[35993]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 858 MiB/s wr, 2.08k op/s 2024-03-21T09:02:12.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:12 smithi100 ceph-mon[36306]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 858 MiB/s wr, 2.08k op/s 2024-03-21T09:02:12.662 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:12 smithi087 ceph-mon[32385]: pgmap v504: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 858 MiB/s wr, 2.08k op/s 2024-03-21T09:02:14.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:14 smithi105 ceph-mon[35993]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1002 MiB/s wr, 2.22k op/s 2024-03-21T09:02:14.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:14 smithi100 ceph-mon[36306]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1002 MiB/s wr, 2.22k op/s 2024-03-21T09:02:14.661 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:14 smithi087 ceph-mon[32385]: pgmap v505: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1002 MiB/s wr, 2.22k op/s 2024-03-21T09:02:16.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:16 smithi100 ceph-mon[36306]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 831 MiB/s wr, 1.86k op/s 2024-03-21T09:02:16.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:16 smithi087 ceph-mon[32385]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 831 MiB/s wr, 1.86k op/s 2024-03-21T09:02:16.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:16 smithi105 ceph-mon[35993]: pgmap v506: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 831 MiB/s wr, 1.86k op/s 2024-03-21T09:02:18.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:18 smithi087 ceph-mon[32385]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 849 MiB/s wr, 2.01k op/s 2024-03-21T09:02:18.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:18 smithi105 ceph-mon[35993]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 849 MiB/s wr, 2.01k op/s 2024-03-21T09:02:18.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:18 smithi100 ceph-mon[36306]: pgmap v507: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 849 MiB/s wr, 2.01k op/s 2024-03-21T09:02:20.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:20 smithi087 ceph-mon[32385]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 934 MiB/s wr, 2.34k op/s 2024-03-21T09:02:20.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:20 smithi105 ceph-mon[35993]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 934 MiB/s wr, 2.34k op/s 2024-03-21T09:02:20.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:20 smithi100 ceph-mon[36306]: pgmap v508: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 934 MiB/s wr, 2.34k op/s 2024-03-21T09:02:22.673 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:22 smithi087 ceph-mon[32385]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 2.10k op/s 2024-03-21T09:02:22.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:22 smithi105 ceph-mon[35993]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 2.10k op/s 2024-03-21T09:02:22.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:22 smithi100 ceph-mon[36306]: pgmap v509: 97 pgs: 97 active+clean; 10 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 843 MiB/s wr, 2.10k op/s 2024-03-21T09:02:24.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:24 smithi087 ceph-mon[32385]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 991 MiB/s wr, 2.13k op/s 2024-03-21T09:02:24.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:24 smithi105 ceph-mon[35993]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 991 MiB/s wr, 2.13k op/s 2024-03-21T09:02:24.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:24 smithi100 ceph-mon[36306]: pgmap v510: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 991 MiB/s wr, 2.13k op/s 2024-03-21T09:02:26.677 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:26 smithi087 ceph-mon[32385]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 812 MiB/s wr, 1.87k op/s 2024-03-21T09:02:26.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:26 smithi105 ceph-mon[35993]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 812 MiB/s wr, 1.87k op/s 2024-03-21T09:02:26.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:26 smithi100 ceph-mon[36306]: pgmap v511: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 812 MiB/s wr, 1.87k op/s 2024-03-21T09:02:28.666 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:28 smithi087 ceph-mon[32385]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 828 MiB/s wr, 1.88k op/s 2024-03-21T09:02:28.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:28 smithi105 ceph-mon[35993]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 828 MiB/s wr, 1.88k op/s 2024-03-21T09:02:28.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:28 smithi100 ceph-mon[36306]: pgmap v512: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 828 MiB/s wr, 1.88k op/s 2024-03-21T09:02:30.660 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:30 smithi087 ceph-mon[32385]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 922 MiB/s wr, 1.97k op/s 2024-03-21T09:02:30.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:30 smithi105 ceph-mon[35993]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 922 MiB/s wr, 1.97k op/s 2024-03-21T09:02:30.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:30 smithi100 ceph-mon[36306]: pgmap v513: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 922 MiB/s wr, 1.97k op/s 2024-03-21T09:02:32.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:32 smithi087 ceph-mon[32385]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 855 MiB/s wr, 1.57k op/s 2024-03-21T09:02:32.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:32 smithi105 ceph-mon[35993]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 855 MiB/s wr, 1.57k op/s 2024-03-21T09:02:32.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:32 smithi100 ceph-mon[36306]: pgmap v514: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 855 MiB/s wr, 1.57k op/s 2024-03-21T09:02:34.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:34 smithi105 ceph-mon[35993]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 972 MiB/s wr, 1.90k op/s 2024-03-21T09:02:34.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:34 smithi100 ceph-mon[36306]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 972 MiB/s wr, 1.90k op/s 2024-03-21T09:02:34.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:34 smithi087 ceph-mon[32385]: pgmap v515: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 972 MiB/s wr, 1.90k op/s 2024-03-21T09:02:36.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:36 smithi105 ceph-mon[35993]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 825 MiB/s wr, 1.90k op/s 2024-03-21T09:02:36.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:36 smithi100 ceph-mon[36306]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 825 MiB/s wr, 1.90k op/s 2024-03-21T09:02:36.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:36 smithi087 ceph-mon[32385]: pgmap v516: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 825 MiB/s wr, 1.90k op/s 2024-03-21T09:02:38.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:38 smithi105 ceph-mon[35993]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 841 MiB/s wr, 1.94k op/s 2024-03-21T09:02:38.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:38 smithi100 ceph-mon[36306]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 841 MiB/s wr, 1.94k op/s 2024-03-21T09:02:38.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:38 smithi087 ceph-mon[32385]: pgmap v517: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 841 MiB/s wr, 1.94k op/s 2024-03-21T09:02:40.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:40 smithi100 ceph-mon[36306]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 945 MiB/s wr, 2.02k op/s 2024-03-21T09:02:40.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:40 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:40.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:40 smithi087 ceph-mon[32385]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 945 MiB/s wr, 2.02k op/s 2024-03-21T09:02:40.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:40 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:40 smithi105 ceph-mon[35993]: pgmap v518: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 945 MiB/s wr, 2.02k op/s 2024-03-21T09:02:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:40 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:02:42.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:42 smithi100 ceph-mon[36306]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 878 MiB/s wr, 1.84k op/s 2024-03-21T09:02:42.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:42 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:42.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:42 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:42.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:42 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:02:42.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:42 smithi087 ceph-mon[32385]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 878 MiB/s wr, 1.84k op/s 2024-03-21T09:02:42.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:42 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:42.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:42 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:42.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:42 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:02:42.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:42 smithi105 ceph-mon[35993]: pgmap v519: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 878 MiB/s wr, 1.84k op/s 2024-03-21T09:02:42.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:42 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:02:42.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:42 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:02:42.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:42 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:02:44.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:44 smithi100 ceph-mon[36306]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 975 MiB/s wr, 2.02k op/s 2024-03-21T09:02:44.784 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:44 smithi105 ceph-mon[35993]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 975 MiB/s wr, 2.02k op/s 2024-03-21T09:02:44.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:44 smithi087 ceph-mon[32385]: pgmap v520: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 975 MiB/s wr, 2.02k op/s 2024-03-21T09:02:46.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:46 smithi087 ceph-mon[32385]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 871 MiB/s wr, 1.61k op/s 2024-03-21T09:02:46.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:46 smithi105 ceph-mon[35993]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 871 MiB/s wr, 1.61k op/s 2024-03-21T09:02:47.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:46 smithi100 ceph-mon[36306]: pgmap v521: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 871 MiB/s wr, 1.61k op/s 2024-03-21T09:02:48.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:48 smithi087 ceph-mon[32385]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 853 MiB/s wr, 1.49k op/s 2024-03-21T09:02:48.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:48 smithi105 ceph-mon[35993]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 853 MiB/s wr, 1.49k op/s 2024-03-21T09:02:49.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:48 smithi100 ceph-mon[36306]: pgmap v522: 97 pgs: 97 active+clean; 10 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 853 MiB/s wr, 1.49k op/s 2024-03-21T09:02:50.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:50 smithi087 ceph-mon[32385]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 952 MiB/s wr, 1.68k op/s 2024-03-21T09:02:50.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:50 smithi105 ceph-mon[35993]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 952 MiB/s wr, 1.68k op/s 2024-03-21T09:02:51.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:50 smithi100 ceph-mon[36306]: pgmap v523: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 952 MiB/s wr, 1.68k op/s 2024-03-21T09:02:51.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:51 smithi087 ceph-mon[32385]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 MiB/s wr, 1.77k op/s 2024-03-21T09:02:51.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:51 smithi105 ceph-mon[35993]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 MiB/s wr, 1.77k op/s 2024-03-21T09:02:52.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:51 smithi100 ceph-mon[36306]: pgmap v524: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 876 MiB/s wr, 1.77k op/s 2024-03-21T09:02:54.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:54 smithi100 ceph-mon[36306]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 956 MiB/s wr, 2.34k op/s 2024-03-21T09:02:54.614 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:54 smithi087 ceph-mon[32385]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 956 MiB/s wr, 2.34k op/s 2024-03-21T09:02:54.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:54 smithi105 ceph-mon[35993]: pgmap v525: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 956 MiB/s wr, 2.34k op/s 2024-03-21T09:02:56.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:56 smithi100 ceph-mon[36306]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 828 MiB/s wr, 2.27k op/s 2024-03-21T09:02:56.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:56 smithi087 ceph-mon[32385]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 828 MiB/s wr, 2.27k op/s 2024-03-21T09:02:56.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:56 smithi105 ceph-mon[35993]: pgmap v526: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 828 MiB/s wr, 2.27k op/s 2024-03-21T09:02:58.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:02:58 smithi100 ceph-mon[36306]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 824 MiB/s wr, 2.38k op/s 2024-03-21T09:02:58.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:02:58 smithi087 ceph-mon[32385]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 824 MiB/s wr, 2.38k op/s 2024-03-21T09:02:58.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:02:58 smithi105 ceph-mon[35993]: pgmap v527: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 824 MiB/s wr, 2.38k op/s 2024-03-21T09:03:00.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:00 smithi087 ceph-mon[32385]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 927 MiB/s wr, 2.70k op/s 2024-03-21T09:03:00.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:00 smithi105 ceph-mon[35993]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 927 MiB/s wr, 2.70k op/s 2024-03-21T09:03:00.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:00 smithi100 ceph-mon[36306]: pgmap v528: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 927 MiB/s wr, 2.70k op/s 2024-03-21T09:03:02.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:02 smithi087 ceph-mon[32385]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 858 MiB/s wr, 2.49k op/s 2024-03-21T09:03:02.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:02 smithi105 ceph-mon[35993]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 858 MiB/s wr, 2.49k op/s 2024-03-21T09:03:02.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:02 smithi100 ceph-mon[36306]: pgmap v529: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 858 MiB/s wr, 2.49k op/s 2024-03-21T09:03:04.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:04 smithi087 ceph-mon[32385]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 930 MiB/s wr, 2.72k op/s 2024-03-21T09:03:04.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:04 smithi105 ceph-mon[35993]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 930 MiB/s wr, 2.72k op/s 2024-03-21T09:03:04.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:04 smithi100 ceph-mon[36306]: pgmap v530: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 930 MiB/s wr, 2.72k op/s 2024-03-21T09:03:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:06 smithi105 ceph-mon[35993]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 817 MiB/s wr, 2.29k op/s 2024-03-21T09:03:06.518 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:06 smithi100 ceph-mon[36306]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 817 MiB/s wr, 2.29k op/s 2024-03-21T09:03:06.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:06 smithi087 ceph-mon[32385]: pgmap v531: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 817 MiB/s wr, 2.29k op/s 2024-03-21T09:03:08.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:08 smithi105 ceph-mon[35993]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 841 MiB/s wr, 2.43k op/s 2024-03-21T09:03:08.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:08 smithi100 ceph-mon[36306]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 841 MiB/s wr, 2.43k op/s 2024-03-21T09:03:08.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:08 smithi087 ceph-mon[32385]: pgmap v532: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 841 MiB/s wr, 2.43k op/s 2024-03-21T09:03:10.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:10 smithi105 ceph-mon[35993]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 2.77k op/s 2024-03-21T09:03:10.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:10 smithi100 ceph-mon[36306]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 2.77k op/s 2024-03-21T09:03:10.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:10 smithi087 ceph-mon[32385]: pgmap v533: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 925 MiB/s wr, 2.77k op/s 2024-03-21T09:03:12.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:12 smithi105 ceph-mon[35993]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 2.49k op/s 2024-03-21T09:03:12.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:12 smithi100 ceph-mon[36306]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 2.49k op/s 2024-03-21T09:03:12.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:12 smithi087 ceph-mon[32385]: pgmap v534: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 794 MiB/s wr, 2.49k op/s 2024-03-21T09:03:14.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:14 smithi100 ceph-mon[36306]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 900 MiB/s wr, 3.27k op/s 2024-03-21T09:03:14.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:14 smithi087 ceph-mon[32385]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 900 MiB/s wr, 3.27k op/s 2024-03-21T09:03:14.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:14 smithi105 ceph-mon[35993]: pgmap v535: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 900 MiB/s wr, 3.27k op/s 2024-03-21T09:03:16.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:16 smithi087 ceph-mon[32385]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 799 MiB/s wr, 2.92k op/s 2024-03-21T09:03:16.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:16 smithi105 ceph-mon[35993]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 799 MiB/s wr, 2.92k op/s 2024-03-21T09:03:16.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:16 smithi100 ceph-mon[36306]: pgmap v536: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 799 MiB/s wr, 2.92k op/s 2024-03-21T09:03:18.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:18 smithi087 ceph-mon[32385]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 822 MiB/s wr, 2.72k op/s 2024-03-21T09:03:18.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:18 smithi105 ceph-mon[35993]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 822 MiB/s wr, 2.72k op/s 2024-03-21T09:03:18.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:18 smithi100 ceph-mon[36306]: pgmap v537: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 822 MiB/s wr, 2.72k op/s 2024-03-21T09:03:20.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:20 smithi087 ceph-mon[32385]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 911 MiB/s wr, 2.49k op/s 2024-03-21T09:03:20.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:20 smithi105 ceph-mon[35993]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 911 MiB/s wr, 2.49k op/s 2024-03-21T09:03:20.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:20 smithi100 ceph-mon[36306]: pgmap v538: 97 pgs: 97 active+clean; 10 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 911 MiB/s wr, 2.49k op/s 2024-03-21T09:03:22.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:22 smithi087 ceph-mon[32385]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 1.96k op/s 2024-03-21T09:03:22.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:22 smithi105 ceph-mon[35993]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 1.96k op/s 2024-03-21T09:03:22.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:22 smithi100 ceph-mon[36306]: pgmap v539: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 790 MiB/s wr, 1.96k op/s 2024-03-21T09:03:24.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:24 smithi087 ceph-mon[32385]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 978 MiB/s wr, 2.35k op/s 2024-03-21T09:03:24.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:24 smithi105 ceph-mon[35993]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 978 MiB/s wr, 2.35k op/s 2024-03-21T09:03:24.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:24 smithi100 ceph-mon[36306]: pgmap v540: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 978 MiB/s wr, 2.35k op/s 2024-03-21T09:03:26.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:26 smithi087 ceph-mon[32385]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 845 MiB/s wr, 1.65k op/s 2024-03-21T09:03:26.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:26 smithi105 ceph-mon[35993]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 845 MiB/s wr, 1.65k op/s 2024-03-21T09:03:26.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:26 smithi100 ceph-mon[36306]: pgmap v541: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 845 MiB/s wr, 1.65k op/s 2024-03-21T09:03:28.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:28 smithi087 ceph-mon[32385]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 871 MiB/s wr, 1.74k op/s 2024-03-21T09:03:28.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:28 smithi105 ceph-mon[35993]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 871 MiB/s wr, 1.74k op/s 2024-03-21T09:03:28.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:28 smithi100 ceph-mon[36306]: pgmap v542: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 871 MiB/s wr, 1.74k op/s 2024-03-21T09:03:30.657 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:30 smithi087 ceph-mon[32385]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 938 MiB/s wr, 2.25k op/s 2024-03-21T09:03:30.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:30 smithi105 ceph-mon[35993]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 938 MiB/s wr, 2.25k op/s 2024-03-21T09:03:30.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:30 smithi100 ceph-mon[36306]: pgmap v543: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 938 MiB/s wr, 2.25k op/s 2024-03-21T09:03:32.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:32 smithi087 ceph-mon[32385]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 808 MiB/s wr, 2.21k op/s 2024-03-21T09:03:32.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:32 smithi105 ceph-mon[35993]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 808 MiB/s wr, 2.21k op/s 2024-03-21T09:03:32.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:32 smithi100 ceph-mon[36306]: pgmap v544: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 808 MiB/s wr, 2.21k op/s 2024-03-21T09:03:34.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:34 smithi087 ceph-mon[32385]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 983 MiB/s wr, 2.47k op/s 2024-03-21T09:03:34.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:34 smithi105 ceph-mon[35993]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 983 MiB/s wr, 2.47k op/s 2024-03-21T09:03:34.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:34 smithi100 ceph-mon[36306]: pgmap v545: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 983 MiB/s wr, 2.47k op/s 2024-03-21T09:03:36.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:36 smithi087 ceph-mon[32385]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 822 MiB/s wr, 2.08k op/s 2024-03-21T09:03:36.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:36 smithi105 ceph-mon[35993]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 822 MiB/s wr, 2.08k op/s 2024-03-21T09:03:36.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:36 smithi100 ceph-mon[36306]: pgmap v546: 97 pgs: 97 active+clean; 10 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 822 MiB/s wr, 2.08k op/s 2024-03-21T09:03:38.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:38 smithi105 ceph-mon[35993]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 2.16k op/s 2024-03-21T09:03:38.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:38 smithi100 ceph-mon[36306]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 2.16k op/s 2024-03-21T09:03:38.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:38 smithi087 ceph-mon[32385]: pgmap v547: 97 pgs: 97 active+clean; 10 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 842 MiB/s wr, 2.16k op/s 2024-03-21T09:03:40.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:40 smithi105 ceph-mon[35993]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 919 MiB/s wr, 2.36k op/s 2024-03-21T09:03:40.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:40 smithi100 ceph-mon[36306]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 919 MiB/s wr, 2.36k op/s 2024-03-21T09:03:40.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:40 smithi087 ceph-mon[32385]: pgmap v548: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 919 MiB/s wr, 2.36k op/s 2024-03-21T09:03:42.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:42 smithi105 ceph-mon[35993]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 MiB/s wr, 1.81k op/s 2024-03-21T09:03:42.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:42 smithi100 ceph-mon[36306]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 MiB/s wr, 1.81k op/s 2024-03-21T09:03:42.914 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:42 smithi087 ceph-mon[32385]: pgmap v549: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 809 MiB/s wr, 1.81k op/s 2024-03-21T09:03:43.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:43 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:43.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:43 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:43.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:43 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:03:44.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:44 smithi100 ceph-mon[36306]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 993 MiB/s wr, 2.20k op/s 2024-03-21T09:03:44.883 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:44 smithi105 ceph-mon[35993]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 993 MiB/s wr, 2.20k op/s 2024-03-21T09:03:44.915 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:44 smithi087 ceph-mon[32385]: pgmap v550: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 993 MiB/s wr, 2.20k op/s 2024-03-21T09:03:46.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 808 MiB/s wr, 1.93k op/s 2024-03-21T09:03:46.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:46.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:46 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:03:46.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 808 MiB/s wr, 1.93k op/s 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:46.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:46 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: pgmap v551: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 808 MiB/s wr, 1.93k op/s 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:03:47.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:46 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:03:47.910 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:47 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi087 to 176.2M 2024-03-21T09:03:47.910 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:47 smithi087 ceph-mon[32385]: Unable to set osd_memory_target on smithi087 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-03-21T09:03:47.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:47 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi087 to 176.2M 2024-03-21T09:03:47.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:47 smithi105 ceph-mon[35993]: Unable to set osd_memory_target on smithi087 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-03-21T09:03:48.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:47 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi087 to 176.2M 2024-03-21T09:03:48.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:47 smithi100 ceph-mon[36306]: Unable to set osd_memory_target on smithi087 to 184760934: error parsing value: Value '184760934' is below minimum 939524096 2024-03-21T09:03:48.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:48 smithi087 ceph-mon[32385]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 881 MiB/s wr, 2.04k op/s 2024-03-21T09:03:48.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:48 smithi105 ceph-mon[35993]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 881 MiB/s wr, 2.04k op/s 2024-03-21T09:03:49.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:48 smithi100 ceph-mon[36306]: pgmap v552: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 881 MiB/s wr, 2.04k op/s 2024-03-21T09:03:50.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:50 smithi087 ceph-mon[32385]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 941 MiB/s wr, 2.05k op/s 2024-03-21T09:03:50.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:50 smithi105 ceph-mon[35993]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 941 MiB/s wr, 2.05k op/s 2024-03-21T09:03:51.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:50 smithi100 ceph-mon[36306]: pgmap v553: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 941 MiB/s wr, 2.05k op/s 2024-03-21T09:03:52.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:52 smithi087 ceph-mon[32385]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 813 MiB/s wr, 1.73k op/s 2024-03-21T09:03:52.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:52 smithi105 ceph-mon[35993]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 813 MiB/s wr, 1.73k op/s 2024-03-21T09:03:53.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:52 smithi100 ceph-mon[36306]: pgmap v554: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 813 MiB/s wr, 1.73k op/s 2024-03-21T09:03:53.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:53 smithi087 ceph-mon[32385]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 988 MiB/s wr, 2.26k op/s 2024-03-21T09:03:53.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:53 smithi105 ceph-mon[35993]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 988 MiB/s wr, 2.26k op/s 2024-03-21T09:03:54.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:53 smithi100 ceph-mon[36306]: pgmap v555: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 988 MiB/s wr, 2.26k op/s 2024-03-21T09:03:56.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:56 smithi100 ceph-mon[36306]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 799 MiB/s wr, 1.83k op/s 2024-03-21T09:03:56.611 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:56 smithi087 ceph-mon[32385]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 799 MiB/s wr, 1.83k op/s 2024-03-21T09:03:56.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:56 smithi105 ceph-mon[35993]: pgmap v556: 97 pgs: 97 active+clean; 10 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 799 MiB/s wr, 1.83k op/s 2024-03-21T09:03:58.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:03:58 smithi087 ceph-mon[32385]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 2.08k op/s 2024-03-21T09:03:58.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:03:58 smithi105 ceph-mon[35993]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 2.08k op/s 2024-03-21T09:03:58.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:03:58 smithi100 ceph-mon[36306]: pgmap v557: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 895 MiB/s wr, 2.08k op/s 2024-03-21T09:04:00.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:00 smithi087 ceph-mon[32385]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 MiB/s wr, 2.39k op/s 2024-03-21T09:04:00.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:00 smithi105 ceph-mon[35993]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 MiB/s wr, 2.39k op/s 2024-03-21T09:04:00.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:00 smithi100 ceph-mon[36306]: pgmap v558: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 904 MiB/s wr, 2.39k op/s 2024-03-21T09:04:02.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:02 smithi087 ceph-mon[32385]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 2.24k op/s 2024-03-21T09:04:02.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:02 smithi105 ceph-mon[35993]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 2.24k op/s 2024-03-21T09:04:02.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:02 smithi100 ceph-mon[36306]: pgmap v559: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 2.24k op/s 2024-03-21T09:04:04.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:04 smithi087 ceph-mon[32385]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 976 MiB/s wr, 2.47k op/s 2024-03-21T09:04:04.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:04 smithi105 ceph-mon[35993]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 976 MiB/s wr, 2.47k op/s 2024-03-21T09:04:04.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:04 smithi100 ceph-mon[36306]: pgmap v560: 97 pgs: 97 active+clean; 10 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 976 MiB/s wr, 2.47k op/s 2024-03-21T09:04:06.658 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:06 smithi087 ceph-mon[32385]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 1.89k op/s 2024-03-21T09:04:06.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:06 smithi105 ceph-mon[35993]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 1.89k op/s 2024-03-21T09:04:06.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:06 smithi100 ceph-mon[36306]: pgmap v561: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 793 MiB/s wr, 1.89k op/s 2024-03-21T09:04:08.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:08 smithi087 ceph-mon[32385]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 900 MiB/s wr, 2.04k op/s 2024-03-21T09:04:08.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:08 smithi105 ceph-mon[35993]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 900 MiB/s wr, 2.04k op/s 2024-03-21T09:04:08.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:08 smithi100 ceph-mon[36306]: pgmap v562: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 900 MiB/s wr, 2.04k op/s 2024-03-21T09:04:10.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:10 smithi087 ceph-mon[32385]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 MiB/s wr, 1.97k op/s 2024-03-21T09:04:10.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:10 smithi105 ceph-mon[35993]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 MiB/s wr, 1.97k op/s 2024-03-21T09:04:10.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:10 smithi100 ceph-mon[36306]: pgmap v563: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 918 MiB/s wr, 1.97k op/s 2024-03-21T09:04:12.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:12 smithi087 ceph-mon[32385]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 818 MiB/s wr, 1.51k op/s 2024-03-21T09:04:12.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:12 smithi105 ceph-mon[35993]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 818 MiB/s wr, 1.51k op/s 2024-03-21T09:04:12.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:12 smithi100 ceph-mon[36306]: pgmap v564: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 818 MiB/s wr, 1.51k op/s 2024-03-21T09:04:14.662 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:14 smithi087 ceph-mon[32385]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 993 MiB/s wr, 2.04k op/s 2024-03-21T09:04:14.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:14 smithi105 ceph-mon[35993]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 993 MiB/s wr, 2.04k op/s 2024-03-21T09:04:14.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:14 smithi100 ceph-mon[36306]: pgmap v565: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 993 MiB/s wr, 2.04k op/s 2024-03-21T09:04:16.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:16 smithi087 ceph-mon[32385]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 1.81k op/s 2024-03-21T09:04:16.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:16 smithi105 ceph-mon[35993]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 1.81k op/s 2024-03-21T09:04:16.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:16 smithi100 ceph-mon[36306]: pgmap v566: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 1.81k op/s 2024-03-21T09:04:18.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:18 smithi105 ceph-mon[35993]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 2.03k op/s 2024-03-21T09:04:18.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:18 smithi100 ceph-mon[36306]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 2.03k op/s 2024-03-21T09:04:18.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:18 smithi087 ceph-mon[32385]: pgmap v567: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 2.03k op/s 2024-03-21T09:04:20.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:20 smithi105 ceph-mon[35993]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 2.14k op/s 2024-03-21T09:04:20.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:20 smithi100 ceph-mon[36306]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 2.14k op/s 2024-03-21T09:04:20.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:20 smithi087 ceph-mon[32385]: pgmap v568: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 896 MiB/s wr, 2.14k op/s 2024-03-21T09:04:22.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:22 smithi105 ceph-mon[35993]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 794 MiB/s wr, 1.92k op/s 2024-03-21T09:04:22.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:22 smithi100 ceph-mon[36306]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 794 MiB/s wr, 1.92k op/s 2024-03-21T09:04:22.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:22 smithi087 ceph-mon[32385]: pgmap v569: 97 pgs: 97 active+clean; 10 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 794 MiB/s wr, 1.92k op/s 2024-03-21T09:04:24.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:24 smithi105 ceph-mon[35993]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 982 MiB/s wr, 2.50k op/s 2024-03-21T09:04:24.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:24 smithi100 ceph-mon[36306]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 982 MiB/s wr, 2.50k op/s 2024-03-21T09:04:24.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:24 smithi087 ceph-mon[32385]: pgmap v570: 97 pgs: 97 active+clean; 10 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 982 MiB/s wr, 2.50k op/s 2024-03-21T09:04:26.714 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:26 smithi087 ceph-mon[32385]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 791 MiB/s wr, 1.98k op/s 2024-03-21T09:04:26.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:26 smithi105 ceph-mon[35993]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 791 MiB/s wr, 1.98k op/s 2024-03-21T09:04:26.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:26 smithi100 ceph-mon[36306]: pgmap v571: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 791 MiB/s wr, 1.98k op/s 2024-03-21T09:04:28.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:28 smithi100 ceph-mon[36306]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 2.07k op/s 2024-03-21T09:04:28.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:28 smithi087 ceph-mon[32385]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 2.07k op/s 2024-03-21T09:04:28.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:28 smithi105 ceph-mon[35993]: pgmap v572: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 854 MiB/s wr, 2.07k op/s 2024-03-21T09:04:30.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:30 smithi087 ceph-mon[32385]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 950 MiB/s wr, 2.04k op/s 2024-03-21T09:04:30.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:30 smithi105 ceph-mon[35993]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 950 MiB/s wr, 2.04k op/s 2024-03-21T09:04:31.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:30 smithi100 ceph-mon[36306]: pgmap v573: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 950 MiB/s wr, 2.04k op/s 2024-03-21T09:04:32.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:32 smithi087 ceph-mon[32385]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 804 MiB/s wr, 1.79k op/s 2024-03-21T09:04:32.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:32 smithi105 ceph-mon[35993]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 804 MiB/s wr, 1.79k op/s 2024-03-21T09:04:33.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:32 smithi100 ceph-mon[36306]: pgmap v574: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 804 MiB/s wr, 1.79k op/s 2024-03-21T09:04:34.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:34 smithi087 ceph-mon[32385]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 984 MiB/s wr, 2.45k op/s 2024-03-21T09:04:34.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:34 smithi105 ceph-mon[35993]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 984 MiB/s wr, 2.45k op/s 2024-03-21T09:04:35.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:34 smithi100 ceph-mon[36306]: pgmap v575: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 984 MiB/s wr, 2.45k op/s 2024-03-21T09:04:36.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:36 smithi087 ceph-mon[32385]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 782 MiB/s wr, 1.90k op/s 2024-03-21T09:04:36.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:36 smithi105 ceph-mon[35993]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 782 MiB/s wr, 1.90k op/s 2024-03-21T09:04:37.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:36 smithi100 ceph-mon[36306]: pgmap v576: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 782 MiB/s wr, 1.90k op/s 2024-03-21T09:04:38.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:38 smithi087 ceph-mon[32385]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 844 MiB/s wr, 1.90k op/s 2024-03-21T09:04:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:38 smithi105 ceph-mon[35993]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 844 MiB/s wr, 1.90k op/s 2024-03-21T09:04:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:38 smithi100 ceph-mon[36306]: pgmap v577: 97 pgs: 97 active+clean; 10 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 844 MiB/s wr, 1.90k op/s 2024-03-21T09:04:40.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:40 smithi087 ceph-mon[32385]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 956 MiB/s wr, 2.19k op/s 2024-03-21T09:04:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:40 smithi105 ceph-mon[35993]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 956 MiB/s wr, 2.19k op/s 2024-03-21T09:04:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:40 smithi100 ceph-mon[36306]: pgmap v578: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 956 MiB/s wr, 2.19k op/s 2024-03-21T09:04:42.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:42 smithi087 ceph-mon[32385]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 MiB/s wr, 2.17k op/s 2024-03-21T09:04:42.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:42 smithi105 ceph-mon[35993]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 MiB/s wr, 2.17k op/s 2024-03-21T09:04:43.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:42 smithi100 ceph-mon[36306]: pgmap v579: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 811 MiB/s wr, 2.17k op/s 2024-03-21T09:04:44.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:44 smithi087 ceph-mon[32385]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 948 MiB/s wr, 2.67k op/s 2024-03-21T09:04:44.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:44 smithi105 ceph-mon[35993]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 948 MiB/s wr, 2.67k op/s 2024-03-21T09:04:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:44 smithi100 ceph-mon[36306]: pgmap v580: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 948 MiB/s wr, 2.67k op/s 2024-03-21T09:04:46.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:46 smithi087 ceph-mon[32385]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 785 MiB/s wr, 2.26k op/s 2024-03-21T09:04:46.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:46 smithi105 ceph-mon[35993]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 785 MiB/s wr, 2.26k op/s 2024-03-21T09:04:47.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:46 smithi100 ceph-mon[36306]: pgmap v581: 97 pgs: 97 active+clean; 10 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 785 MiB/s wr, 2.26k op/s 2024-03-21T09:04:48.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:47 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:48.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:47 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:48.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:47 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 MiB/s wr, 2.47k op/s 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: Unable to set osd_memory_target on smithi100 to 182573260: error parsing value: Value '182573260' is below minimum 939524096 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-21T09:04:49.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:48 smithi087 ceph-mon[32385]: Unable to set osd_memory_target on smithi105 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 MiB/s wr, 2.47k op/s 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: Unable to set osd_memory_target on smithi100 to 182573260: error parsing value: Value '182573260' is below minimum 939524096 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-21T09:04:49.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:48 smithi105 ceph-mon[35993]: Unable to set osd_memory_target on smithi105 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: pgmap v582: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 847 MiB/s wr, 2.47k op/s 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi100 to 174.1M 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: Unable to set osd_memory_target on smithi100 to 182573260: error parsing value: Value '182573260' is below minimum 939524096 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-03-21T09:04:49.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: Adjusting osd_memory_target on smithi105 to 174.1M 2024-03-21T09:04:49.267 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:48 smithi100 ceph-mon[36306]: Unable to set osd_memory_target on smithi105 to 182632038: error parsing value: Value '182632038' is below minimum 939524096 2024-03-21T09:04:50.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:49 smithi087 ceph-mon[32385]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 862 MiB/s wr, 2.78k op/s 2024-03-21T09:04:50.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:49 smithi105 ceph-mon[35993]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 862 MiB/s wr, 2.78k op/s 2024-03-21T09:04:50.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:49 smithi100 ceph-mon[36306]: pgmap v583: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 862 MiB/s wr, 2.78k op/s 2024-03-21T09:04:52.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:52 smithi087 ceph-mon[32385]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 781 MiB/s wr, 2.53k op/s 2024-03-21T09:04:52.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:52.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:52.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:04:52.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:52 smithi105 ceph-mon[35993]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 781 MiB/s wr, 2.53k op/s 2024-03-21T09:04:52.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:52.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:52.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:04:52.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:52 smithi100 ceph-mon[36306]: pgmap v584: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 781 MiB/s wr, 2.53k op/s 2024-03-21T09:04:52.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:04:52.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:04:52.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:04:54.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:54 smithi087 ceph-mon[32385]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 917 MiB/s wr, 2.54k op/s 2024-03-21T09:04:54.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:54 smithi105 ceph-mon[35993]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 917 MiB/s wr, 2.54k op/s 2024-03-21T09:04:54.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:54 smithi100 ceph-mon[36306]: pgmap v585: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 917 MiB/s wr, 2.54k op/s 2024-03-21T09:04:56.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:56 smithi087 ceph-mon[32385]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 2.05k op/s 2024-03-21T09:04:56.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:56 smithi105 ceph-mon[35993]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 2.05k op/s 2024-03-21T09:04:56.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:56 smithi100 ceph-mon[36306]: pgmap v586: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 823 MiB/s wr, 2.05k op/s 2024-03-21T09:04:58.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:04:58 smithi105 ceph-mon[35993]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 862 MiB/s wr, 1.93k op/s 2024-03-21T09:04:58.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:04:58 smithi100 ceph-mon[36306]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 862 MiB/s wr, 1.93k op/s 2024-03-21T09:04:58.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:04:58 smithi087 ceph-mon[32385]: pgmap v587: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 862 MiB/s wr, 1.93k op/s 2024-03-21T09:05:00.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:00 smithi105 ceph-mon[35993]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 877 MiB/s wr, 1.85k op/s 2024-03-21T09:05:00.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:00 smithi100 ceph-mon[36306]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 877 MiB/s wr, 1.85k op/s 2024-03-21T09:05:00.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:00 smithi087 ceph-mon[32385]: pgmap v588: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 877 MiB/s wr, 1.85k op/s 2024-03-21T09:05:02.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:02 smithi105 ceph-mon[35993]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 882 MiB/s wr, 1.78k op/s 2024-03-21T09:05:02.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:02 smithi100 ceph-mon[36306]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 882 MiB/s wr, 1.78k op/s 2024-03-21T09:05:02.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:02 smithi087 ceph-mon[32385]: pgmap v589: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 882 MiB/s wr, 1.78k op/s 2024-03-21T09:05:04.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:04 smithi100 ceph-mon[36306]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 929 MiB/s wr, 1.87k op/s 2024-03-21T09:05:04.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:04 smithi087 ceph-mon[32385]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 929 MiB/s wr, 1.87k op/s 2024-03-21T09:05:04.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:04 smithi105 ceph-mon[35993]: pgmap v590: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 929 MiB/s wr, 1.87k op/s 2024-03-21T09:05:06.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:06 smithi087 ceph-mon[32385]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 840 MiB/s wr, 1.72k op/s 2024-03-21T09:05:06.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:06 smithi105 ceph-mon[35993]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 840 MiB/s wr, 1.72k op/s 2024-03-21T09:05:07.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:06 smithi100 ceph-mon[36306]: pgmap v591: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 840 MiB/s wr, 1.72k op/s 2024-03-21T09:05:08.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:08 smithi087 ceph-mon[32385]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 853 MiB/s wr, 1.73k op/s 2024-03-21T09:05:08.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:08 smithi105 ceph-mon[35993]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 853 MiB/s wr, 1.73k op/s 2024-03-21T09:05:09.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:08 smithi100 ceph-mon[36306]: pgmap v592: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 853 MiB/s wr, 1.73k op/s 2024-03-21T09:05:10.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:10 smithi087 ceph-mon[32385]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 858 MiB/s wr, 1.75k op/s 2024-03-21T09:05:10.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:10 smithi105 ceph-mon[35993]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 858 MiB/s wr, 1.75k op/s 2024-03-21T09:05:11.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:10 smithi100 ceph-mon[36306]: pgmap v593: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 858 MiB/s wr, 1.75k op/s 2024-03-21T09:05:12.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:12 smithi087 ceph-mon[32385]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 870 MiB/s wr, 1.70k op/s 2024-03-21T09:05:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:12 smithi105 ceph-mon[35993]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 870 MiB/s wr, 1.70k op/s 2024-03-21T09:05:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:12 smithi100 ceph-mon[36306]: pgmap v594: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 870 MiB/s wr, 1.70k op/s 2024-03-21T09:05:14.912 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:14 smithi087 ceph-mon[32385]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 949 MiB/s wr, 1.62k op/s 2024-03-21T09:05:14.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:14 smithi105 ceph-mon[35993]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 949 MiB/s wr, 1.62k op/s 2024-03-21T09:05:15.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:14 smithi100 ceph-mon[36306]: pgmap v595: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 949 MiB/s wr, 1.62k op/s 2024-03-21T09:05:16.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:16 smithi100 ceph-mon[36306]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 MiB/s wr, 1.37k op/s 2024-03-21T09:05:16.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:16 smithi087 ceph-mon[32385]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 MiB/s wr, 1.37k op/s 2024-03-21T09:05:16.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:16 smithi105 ceph-mon[35993]: pgmap v596: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 870 MiB/s wr, 1.37k op/s 2024-03-21T09:05:18.656 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:18 smithi087 ceph-mon[32385]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.47k op/s 2024-03-21T09:05:18.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:18 smithi105 ceph-mon[35993]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.47k op/s 2024-03-21T09:05:18.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:18 smithi100 ceph-mon[36306]: pgmap v597: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 881 MiB/s wr, 1.47k op/s 2024-03-21T09:05:20.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:20 smithi087 ceph-mon[32385]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 891 MiB/s wr, 1.58k op/s 2024-03-21T09:05:20.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:20 smithi105 ceph-mon[35993]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 891 MiB/s wr, 1.58k op/s 2024-03-21T09:05:20.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:20 smithi100 ceph-mon[36306]: pgmap v598: 97 pgs: 97 active+clean; 10 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 891 MiB/s wr, 1.58k op/s 2024-03-21T09:05:22.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:22 smithi105 ceph-mon[35993]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 MiB/s wr, 1.81k op/s 2024-03-21T09:05:22.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:22 smithi100 ceph-mon[36306]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 MiB/s wr, 1.81k op/s 2024-03-21T09:05:22.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:22 smithi087 ceph-mon[32385]: pgmap v599: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 896 MiB/s wr, 1.81k op/s 2024-03-21T09:05:24.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:24 smithi105 ceph-mon[35993]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 918 MiB/s wr, 2.00k op/s 2024-03-21T09:05:24.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:24 smithi100 ceph-mon[36306]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 918 MiB/s wr, 2.00k op/s 2024-03-21T09:05:24.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:24 smithi087 ceph-mon[32385]: pgmap v600: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 918 MiB/s wr, 2.00k op/s 2024-03-21T09:05:26.708 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:26 smithi087 ceph-mon[32385]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 MiB/s wr, 1.79k op/s 2024-03-21T09:05:26.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:26 smithi105 ceph-mon[35993]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 MiB/s wr, 1.79k op/s 2024-03-21T09:05:26.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:26 smithi100 ceph-mon[36306]: pgmap v601: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 831 MiB/s wr, 1.79k op/s 2024-03-21T09:05:28.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:28 smithi105 ceph-mon[35993]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.84k op/s 2024-03-21T09:05:28.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:28 smithi100 ceph-mon[36306]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.84k op/s 2024-03-21T09:05:28.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:28 smithi087 ceph-mon[32385]: pgmap v602: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 839 MiB/s wr, 1.84k op/s 2024-03-21T09:05:30.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:30 smithi105 ceph-mon[35993]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 859 MiB/s wr, 1.93k op/s 2024-03-21T09:05:30.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:30 smithi100 ceph-mon[36306]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 859 MiB/s wr, 1.93k op/s 2024-03-21T09:05:30.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:30 smithi087 ceph-mon[32385]: pgmap v603: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 859 MiB/s wr, 1.93k op/s 2024-03-21T09:05:32.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:32 smithi100 ceph-mon[36306]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.96k op/s 2024-03-21T09:05:32.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:32 smithi087 ceph-mon[32385]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.96k op/s 2024-03-21T09:05:32.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:32 smithi105 ceph-mon[35993]: pgmap v604: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 877 MiB/s wr, 1.96k op/s 2024-03-21T09:05:34.916 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:34 smithi087 ceph-mon[32385]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 924 MiB/s wr, 1.78k op/s 2024-03-21T09:05:34.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:34 smithi105 ceph-mon[35993]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 924 MiB/s wr, 1.78k op/s 2024-03-21T09:05:35.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:34 smithi100 ceph-mon[36306]: pgmap v605: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 924 MiB/s wr, 1.78k op/s 2024-03-21T09:05:36.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:36 smithi087 ceph-mon[32385]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 885 MiB/s wr, 1.52k op/s 2024-03-21T09:05:36.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:36 smithi105 ceph-mon[35993]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 885 MiB/s wr, 1.52k op/s 2024-03-21T09:05:37.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:36 smithi100 ceph-mon[36306]: pgmap v606: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 885 MiB/s wr, 1.52k op/s 2024-03-21T09:05:38.909 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:38 smithi087 ceph-mon[32385]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 875 MiB/s wr, 1.59k op/s 2024-03-21T09:05:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:38 smithi105 ceph-mon[35993]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 875 MiB/s wr, 1.59k op/s 2024-03-21T09:05:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:38 smithi100 ceph-mon[36306]: pgmap v607: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 875 MiB/s wr, 1.59k op/s 2024-03-21T09:05:40.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:40 smithi087 ceph-mon[32385]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 890 MiB/s wr, 1.68k op/s 2024-03-21T09:05:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:40 smithi105 ceph-mon[35993]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 890 MiB/s wr, 1.68k op/s 2024-03-21T09:05:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:40 smithi100 ceph-mon[36306]: pgmap v608: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 890 MiB/s wr, 1.68k op/s 2024-03-21T09:05:42.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:42 smithi087 ceph-mon[32385]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 891 MiB/s wr, 1.80k op/s 2024-03-21T09:05:42.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:42 smithi105 ceph-mon[35993]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 891 MiB/s wr, 1.80k op/s 2024-03-21T09:05:43.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:42 smithi100 ceph-mon[36306]: pgmap v609: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 891 MiB/s wr, 1.80k op/s 2024-03-21T09:05:44.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:44 smithi105 ceph-mon[35993]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 927 MiB/s wr, 1.87k op/s 2024-03-21T09:05:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:44 smithi100 ceph-mon[36306]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 927 MiB/s wr, 1.87k op/s 2024-03-21T09:05:45.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:44 smithi087 ceph-mon[32385]: pgmap v610: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 927 MiB/s wr, 1.87k op/s 2024-03-21T09:05:46.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:46 smithi105 ceph-mon[35993]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 868 MiB/s wr, 1.74k op/s 2024-03-21T09:05:47.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:46 smithi100 ceph-mon[36306]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 868 MiB/s wr, 1.74k op/s 2024-03-21T09:05:47.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:46 smithi087 ceph-mon[32385]: pgmap v611: 97 pgs: 97 active+clean; 10 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 868 MiB/s wr, 1.74k op/s 2024-03-21T09:05:48.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:48 smithi105 ceph-mon[35993]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 852 MiB/s wr, 1.86k op/s 2024-03-21T09:05:49.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:48 smithi100 ceph-mon[36306]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 852 MiB/s wr, 1.86k op/s 2024-03-21T09:05:49.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:48 smithi087 ceph-mon[32385]: pgmap v612: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 852 MiB/s wr, 1.86k op/s 2024-03-21T09:05:51.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:50 smithi100 ceph-mon[36306]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 855 MiB/s wr, 2.05k op/s 2024-03-21T09:05:51.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:50 smithi087 ceph-mon[32385]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 855 MiB/s wr, 2.05k op/s 2024-03-21T09:05:51.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:50 smithi105 ceph-mon[35993]: pgmap v613: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 855 MiB/s wr, 2.05k op/s 2024-03-21T09:05:53.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:52 smithi087 ceph-mon[32385]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 843 MiB/s wr, 2.20k op/s 2024-03-21T09:05:53.159 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:52 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:53.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:52 smithi105 ceph-mon[35993]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 843 MiB/s wr, 2.20k op/s 2024-03-21T09:05:53.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:52 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:53.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:52 smithi100 ceph-mon[36306]: pgmap v614: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 843 MiB/s wr, 2.20k op/s 2024-03-21T09:05:53.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:52 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:05:55.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:54 smithi087 ceph-mon[32385]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 917 MiB/s wr, 2.57k op/s 2024-03-21T09:05:55.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:54 smithi105 ceph-mon[35993]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 917 MiB/s wr, 2.57k op/s 2024-03-21T09:05:55.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:54 smithi100 ceph-mon[36306]: pgmap v615: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 917 MiB/s wr, 2.57k op/s 2024-03-21T09:05:56.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:55 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:05:56.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:55 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:05:56.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:05:56.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:05:56.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:55 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:05:57.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:56 smithi087 ceph-mon[32385]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 811 MiB/s wr, 2.44k op/s 2024-03-21T09:05:57.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:56 smithi105 ceph-mon[35993]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 811 MiB/s wr, 2.44k op/s 2024-03-21T09:05:57.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:56 smithi100 ceph-mon[36306]: pgmap v616: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 811 MiB/s wr, 2.44k op/s 2024-03-21T09:05:58.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:05:58 smithi100 ceph-mon[36306]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 836 MiB/s wr, 2.72k op/s 2024-03-21T09:05:58.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:05:58 smithi087 ceph-mon[32385]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 836 MiB/s wr, 2.72k op/s 2024-03-21T09:05:58.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:05:58 smithi105 ceph-mon[35993]: pgmap v617: 97 pgs: 97 active+clean; 10 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 836 MiB/s wr, 2.72k op/s 2024-03-21T09:06:00.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:00 smithi087 ceph-mon[32385]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 846 MiB/s wr, 2.70k op/s 2024-03-21T09:06:00.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:00 smithi105 ceph-mon[35993]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 846 MiB/s wr, 2.70k op/s 2024-03-21T09:06:00.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:00 smithi100 ceph-mon[36306]: pgmap v618: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 846 MiB/s wr, 2.70k op/s 2024-03-21T09:06:02.733 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:02 smithi105 ceph-mon[35993]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 845 MiB/s wr, 2.49k op/s 2024-03-21T09:06:02.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:02 smithi100 ceph-mon[36306]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 845 MiB/s wr, 2.49k op/s 2024-03-21T09:06:02.911 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:02 smithi087 ceph-mon[32385]: pgmap v619: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 845 MiB/s wr, 2.49k op/s 2024-03-21T09:06:04.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:04 smithi105 ceph-mon[35993]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 945 MiB/s wr, 2.48k op/s 2024-03-21T09:06:04.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:04 smithi100 ceph-mon[36306]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 945 MiB/s wr, 2.48k op/s 2024-03-21T09:06:04.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:04 smithi087 ceph-mon[32385]: pgmap v620: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 945 MiB/s wr, 2.48k op/s 2024-03-21T09:06:06.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:06 smithi105 ceph-mon[35993]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.91k op/s 2024-03-21T09:06:06.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:06 smithi100 ceph-mon[36306]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.91k op/s 2024-03-21T09:06:06.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:06 smithi087 ceph-mon[32385]: pgmap v621: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.91k op/s 2024-03-21T09:06:08.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:08 smithi100 ceph-mon[36306]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 900 MiB/s wr, 2.01k op/s 2024-03-21T09:06:08.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:08 smithi087 ceph-mon[32385]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 900 MiB/s wr, 2.01k op/s 2024-03-21T09:06:08.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:08 smithi105 ceph-mon[35993]: pgmap v622: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 900 MiB/s wr, 2.01k op/s 2024-03-21T09:06:10.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:10 smithi087 ceph-mon[32385]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 1.85k op/s 2024-03-21T09:06:10.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:10 smithi105 ceph-mon[35993]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 1.85k op/s 2024-03-21T09:06:11.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:10 smithi100 ceph-mon[36306]: pgmap v623: 97 pgs: 97 active+clean; 10 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 876 MiB/s wr, 1.85k op/s 2024-03-21T09:06:12.918 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:12 smithi087 ceph-mon[32385]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 835 MiB/s wr, 1.75k op/s 2024-03-21T09:06:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:12 smithi105 ceph-mon[35993]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 835 MiB/s wr, 1.75k op/s 2024-03-21T09:06:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:12 smithi100 ceph-mon[36306]: pgmap v624: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 835 MiB/s wr, 1.75k op/s 2024-03-21T09:06:14.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:14 smithi087 ceph-mon[32385]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 965 MiB/s wr, 2.38k op/s 2024-03-21T09:06:14.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:14 smithi105 ceph-mon[35993]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 965 MiB/s wr, 2.38k op/s 2024-03-21T09:06:15.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:14 smithi100 ceph-mon[36306]: pgmap v625: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 965 MiB/s wr, 2.38k op/s 2024-03-21T09:06:16.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:16 smithi087 ceph-mon[32385]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 825 MiB/s wr, 2.15k op/s 2024-03-21T09:06:16.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:16 smithi105 ceph-mon[35993]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 825 MiB/s wr, 2.15k op/s 2024-03-21T09:06:17.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:16 smithi100 ceph-mon[36306]: pgmap v626: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 825 MiB/s wr, 2.15k op/s 2024-03-21T09:06:18.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:18 smithi087 ceph-mon[32385]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 2.34k op/s 2024-03-21T09:06:18.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:18 smithi105 ceph-mon[35993]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 2.34k op/s 2024-03-21T09:06:19.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:18 smithi100 ceph-mon[36306]: pgmap v627: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 896 MiB/s wr, 2.34k op/s 2024-03-21T09:06:20.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:20 smithi087 ceph-mon[32385]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 2.14k op/s 2024-03-21T09:06:20.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:20 smithi105 ceph-mon[35993]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 2.14k op/s 2024-03-21T09:06:21.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:20 smithi100 ceph-mon[36306]: pgmap v628: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 863 MiB/s wr, 2.14k op/s 2024-03-21T09:06:22.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:22 smithi105 ceph-mon[35993]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 814 MiB/s wr, 2.05k op/s 2024-03-21T09:06:23.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:22 smithi100 ceph-mon[36306]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 814 MiB/s wr, 2.05k op/s 2024-03-21T09:06:23.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:22 smithi087 ceph-mon[32385]: pgmap v629: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 814 MiB/s wr, 2.05k op/s 2024-03-21T09:06:24.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:24 smithi105 ceph-mon[35993]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 973 MiB/s wr, 2.72k op/s 2024-03-21T09:06:25.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:24 smithi100 ceph-mon[36306]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 973 MiB/s wr, 2.72k op/s 2024-03-21T09:06:25.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:24 smithi087 ceph-mon[32385]: pgmap v630: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 973 MiB/s wr, 2.72k op/s 2024-03-21T09:06:26.977 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:26 smithi087 ceph-mon[32385]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 769 MiB/s wr, 2.07k op/s 2024-03-21T09:06:26.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:26 smithi105 ceph-mon[35993]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 769 MiB/s wr, 2.07k op/s 2024-03-21T09:06:27.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:26 smithi100 ceph-mon[36306]: pgmap v631: 97 pgs: 97 active+clean; 10 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 769 MiB/s wr, 2.07k op/s 2024-03-21T09:06:28.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:28 smithi105 ceph-mon[35993]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.26k op/s 2024-03-21T09:06:29.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:28 smithi100 ceph-mon[36306]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.26k op/s 2024-03-21T09:06:29.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:28 smithi087 ceph-mon[32385]: pgmap v632: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 884 MiB/s wr, 2.26k op/s 2024-03-21T09:06:30.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:30 smithi105 ceph-mon[35993]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 866 MiB/s wr, 2.02k op/s 2024-03-21T09:06:31.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:30 smithi100 ceph-mon[36306]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 866 MiB/s wr, 2.02k op/s 2024-03-21T09:06:31.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:30 smithi087 ceph-mon[32385]: pgmap v633: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 866 MiB/s wr, 2.02k op/s 2024-03-21T09:06:32.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:32 smithi105 ceph-mon[35993]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 811 MiB/s wr, 2.03k op/s 2024-03-21T09:06:33.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:32 smithi100 ceph-mon[36306]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 811 MiB/s wr, 2.03k op/s 2024-03-21T09:06:33.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:32 smithi087 ceph-mon[32385]: pgmap v634: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 811 MiB/s wr, 2.03k op/s 2024-03-21T09:06:35.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:34 smithi100 ceph-mon[36306]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 977 MiB/s wr, 2.37k op/s 2024-03-21T09:06:35.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:34 smithi087 ceph-mon[32385]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 977 MiB/s wr, 2.37k op/s 2024-03-21T09:06:35.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:34 smithi105 ceph-mon[35993]: pgmap v635: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 977 MiB/s wr, 2.37k op/s 2024-03-21T09:06:37.164 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:36 smithi087 ceph-mon[32385]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.57k op/s 2024-03-21T09:06:37.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:36 smithi105 ceph-mon[35993]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.57k op/s 2024-03-21T09:06:37.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:36 smithi100 ceph-mon[36306]: pgmap v636: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.57k op/s 2024-03-21T09:06:39.169 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:38 smithi087 ceph-mon[32385]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 941 MiB/s wr, 1.77k op/s 2024-03-21T09:06:39.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:38 smithi105 ceph-mon[35993]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 941 MiB/s wr, 1.77k op/s 2024-03-21T09:06:39.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:38 smithi100 ceph-mon[36306]: pgmap v637: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 941 MiB/s wr, 1.77k op/s 2024-03-21T09:06:41.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:40 smithi087 ceph-mon[32385]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 879 MiB/s wr, 1.71k op/s 2024-03-21T09:06:41.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:40 smithi105 ceph-mon[35993]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 879 MiB/s wr, 1.71k op/s 2024-03-21T09:06:41.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:40 smithi100 ceph-mon[36306]: pgmap v638: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 879 MiB/s wr, 1.71k op/s 2024-03-21T09:06:43.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:42 smithi087 ceph-mon[32385]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 813 MiB/s wr, 1.79k op/s 2024-03-21T09:06:43.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:42 smithi105 ceph-mon[35993]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 813 MiB/s wr, 1.79k op/s 2024-03-21T09:06:43.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:42 smithi100 ceph-mon[36306]: pgmap v639: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 813 MiB/s wr, 1.79k op/s 2024-03-21T09:06:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:44 smithi087 ceph-mon[32385]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 2.30k op/s 2024-03-21T09:06:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:44 smithi105 ceph-mon[35993]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 2.30k op/s 2024-03-21T09:06:45.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:44 smithi100 ceph-mon[36306]: pgmap v640: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 975 MiB/s wr, 2.30k op/s 2024-03-21T09:06:47.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:46 smithi087 ceph-mon[32385]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 776 MiB/s wr, 1.86k op/s 2024-03-21T09:06:47.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:46 smithi105 ceph-mon[35993]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 776 MiB/s wr, 1.86k op/s 2024-03-21T09:06:47.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:46 smithi100 ceph-mon[36306]: pgmap v641: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 776 MiB/s wr, 1.86k op/s 2024-03-21T09:06:49.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:48 smithi087 ceph-mon[32385]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 935 MiB/s wr, 2.27k op/s 2024-03-21T09:06:49.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:48 smithi105 ceph-mon[35993]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 935 MiB/s wr, 2.27k op/s 2024-03-21T09:06:49.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:48 smithi100 ceph-mon[36306]: pgmap v642: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 935 MiB/s wr, 2.27k op/s 2024-03-21T09:06:51.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:50 smithi087 ceph-mon[32385]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 847 MiB/s wr, 2.23k op/s 2024-03-21T09:06:51.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:50 smithi105 ceph-mon[35993]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 847 MiB/s wr, 2.23k op/s 2024-03-21T09:06:51.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:50 smithi100 ceph-mon[36306]: pgmap v643: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 847 MiB/s wr, 2.23k op/s 2024-03-21T09:06:53.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:52 smithi105 ceph-mon[35993]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 786 MiB/s wr, 2.04k op/s 2024-03-21T09:06:53.266 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:52 smithi100 ceph-mon[36306]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 786 MiB/s wr, 2.04k op/s 2024-03-21T09:06:53.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:52 smithi087 ceph-mon[32385]: pgmap v644: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 786 MiB/s wr, 2.04k op/s 2024-03-21T09:06:55.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:54 smithi105 ceph-mon[35993]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 980 MiB/s wr, 2.59k op/s 2024-03-21T09:06:55.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:54 smithi100 ceph-mon[36306]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 980 MiB/s wr, 2.59k op/s 2024-03-21T09:06:55.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:54 smithi087 ceph-mon[32385]: pgmap v645: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 980 MiB/s wr, 2.59k op/s 2024-03-21T09:06:56.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:56 smithi100 ceph-mon[36306]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 785 MiB/s wr, 2.05k op/s 2024-03-21T09:06:56.308 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:56 smithi087 ceph-mon[32385]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 785 MiB/s wr, 2.05k op/s 2024-03-21T09:06:56.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:56 smithi105 ceph-mon[35993]: pgmap v646: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 785 MiB/s wr, 2.05k op/s 2024-03-21T09:06:57.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:57 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:57.413 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:57 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:57.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:57 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:06:58.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:58 smithi100 ceph-mon[36306]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 924 MiB/s wr, 2.20k op/s 2024-03-21T09:06:58.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:58 smithi087 ceph-mon[32385]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 924 MiB/s wr, 2.20k op/s 2024-03-21T09:06:58.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:58 smithi105 ceph-mon[35993]: pgmap v647: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 924 MiB/s wr, 2.20k op/s 2024-03-21T09:06:59.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:59.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:06:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:06:59.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:59.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:06:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:06:59.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:06:59.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:06:59.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:06:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:07:00.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:00 smithi087 ceph-mon[32385]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.88k op/s 2024-03-21T09:07:00.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:00 smithi105 ceph-mon[35993]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.88k op/s 2024-03-21T09:07:00.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:00 smithi100 ceph-mon[36306]: pgmap v648: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 867 MiB/s wr, 1.88k op/s 2024-03-21T09:07:02.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:02 smithi105 ceph-mon[35993]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.70k op/s 2024-03-21T09:07:02.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:02 smithi100 ceph-mon[36306]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.70k op/s 2024-03-21T09:07:02.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:02 smithi087 ceph-mon[32385]: pgmap v649: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 815 MiB/s wr, 1.70k op/s 2024-03-21T09:07:04.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:04 smithi105 ceph-mon[35993]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1003 MiB/s wr, 2.05k op/s 2024-03-21T09:07:04.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:04 smithi100 ceph-mon[36306]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1003 MiB/s wr, 2.05k op/s 2024-03-21T09:07:04.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:04 smithi087 ceph-mon[32385]: pgmap v650: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 1003 MiB/s wr, 2.05k op/s 2024-03-21T09:07:06.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:06 smithi087 ceph-mon[32385]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 783 MiB/s wr, 1.36k op/s 2024-03-21T09:07:06.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:06 smithi105 ceph-mon[35993]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 783 MiB/s wr, 1.36k op/s 2024-03-21T09:07:06.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:06 smithi100 ceph-mon[36306]: pgmap v651: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 783 MiB/s wr, 1.36k op/s 2024-03-21T09:07:08.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:08 smithi105 ceph-mon[35993]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 919 MiB/s wr, 1.53k op/s 2024-03-21T09:07:08.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:08 smithi100 ceph-mon[36306]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 919 MiB/s wr, 1.53k op/s 2024-03-21T09:07:08.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:08 smithi087 ceph-mon[32385]: pgmap v652: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 919 MiB/s wr, 1.53k op/s 2024-03-21T09:07:10.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:10 smithi105 ceph-mon[35993]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.59k op/s 2024-03-21T09:07:10.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:10 smithi100 ceph-mon[36306]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.59k op/s 2024-03-21T09:07:10.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:10 smithi087 ceph-mon[32385]: pgmap v653: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 887 MiB/s wr, 1.59k op/s 2024-03-21T09:07:12.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:12 smithi105 ceph-mon[35993]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 815 MiB/s wr, 1.67k op/s 2024-03-21T09:07:12.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:12 smithi100 ceph-mon[36306]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 815 MiB/s wr, 1.67k op/s 2024-03-21T09:07:12.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:12 smithi087 ceph-mon[32385]: pgmap v654: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 815 MiB/s wr, 1.67k op/s 2024-03-21T09:07:14.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:14 smithi100 ceph-mon[36306]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1000 MiB/s wr, 2.53k op/s 2024-03-21T09:07:14.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:14 smithi087 ceph-mon[32385]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1000 MiB/s wr, 2.53k op/s 2024-03-21T09:07:14.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:14 smithi105 ceph-mon[35993]: pgmap v655: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1000 MiB/s wr, 2.53k op/s 2024-03-21T09:07:16.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:16 smithi100 ceph-mon[36306]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 MiB/s wr, 2.10k op/s 2024-03-21T09:07:16.784 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:16 smithi105 ceph-mon[35993]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 MiB/s wr, 2.10k op/s 2024-03-21T09:07:16.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:16 smithi087 ceph-mon[32385]: pgmap v656: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 781 MiB/s wr, 2.10k op/s 2024-03-21T09:07:18.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:18 smithi100 ceph-mon[36306]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 882 MiB/s wr, 2.31k op/s 2024-03-21T09:07:18.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:18 smithi087 ceph-mon[32385]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 882 MiB/s wr, 2.31k op/s 2024-03-21T09:07:18.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:18 smithi105 ceph-mon[35993]: pgmap v657: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 882 MiB/s wr, 2.31k op/s 2024-03-21T09:07:20.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:20 smithi100 ceph-mon[36306]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 883 MiB/s wr, 2.40k op/s 2024-03-21T09:07:20.918 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:20 smithi087 ceph-mon[32385]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 883 MiB/s wr, 2.40k op/s 2024-03-21T09:07:20.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:20 smithi105 ceph-mon[35993]: pgmap v658: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 883 MiB/s wr, 2.40k op/s 2024-03-21T09:07:22.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:22 smithi087 ceph-mon[32385]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 793 MiB/s wr, 2.30k op/s 2024-03-21T09:07:22.983 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:22 smithi105 ceph-mon[35993]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 793 MiB/s wr, 2.30k op/s 2024-03-21T09:07:23.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:22 smithi100 ceph-mon[36306]: pgmap v659: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 793 MiB/s wr, 2.30k op/s 2024-03-21T09:07:24.908 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:24 smithi087 ceph-mon[32385]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 990 MiB/s wr, 2.86k op/s 2024-03-21T09:07:24.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:24 smithi105 ceph-mon[35993]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 990 MiB/s wr, 2.86k op/s 2024-03-21T09:07:25.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:24 smithi100 ceph-mon[36306]: pgmap v660: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 MiB/s rd, 990 MiB/s wr, 2.86k op/s 2024-03-21T09:07:26.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:26 smithi087 ceph-mon[32385]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 771 MiB/s wr, 1.98k op/s 2024-03-21T09:07:26.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:26 smithi105 ceph-mon[35993]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 771 MiB/s wr, 1.98k op/s 2024-03-21T09:07:27.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:26 smithi100 ceph-mon[36306]: pgmap v661: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 771 MiB/s wr, 1.98k op/s 2024-03-21T09:07:28.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:28 smithi087 ceph-mon[32385]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 867 MiB/s wr, 2.25k op/s 2024-03-21T09:07:28.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:28 smithi105 ceph-mon[35993]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 867 MiB/s wr, 2.25k op/s 2024-03-21T09:07:29.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:28 smithi100 ceph-mon[36306]: pgmap v662: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 867 MiB/s wr, 2.25k op/s 2024-03-21T09:07:30.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:30 smithi087 ceph-mon[32385]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 901 MiB/s wr, 2.17k op/s 2024-03-21T09:07:30.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:30 smithi105 ceph-mon[35993]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 901 MiB/s wr, 2.17k op/s 2024-03-21T09:07:31.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:30 smithi100 ceph-mon[36306]: pgmap v663: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 901 MiB/s wr, 2.17k op/s 2024-03-21T09:07:32.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:32 smithi087 ceph-mon[32385]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 790 MiB/s wr, 1.93k op/s 2024-03-21T09:07:32.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:32 smithi105 ceph-mon[35993]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 790 MiB/s wr, 1.93k op/s 2024-03-21T09:07:33.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:32 smithi100 ceph-mon[36306]: pgmap v664: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 790 MiB/s wr, 1.93k op/s 2024-03-21T09:07:34.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:34 smithi087 ceph-mon[32385]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 999 MiB/s wr, 2.63k op/s 2024-03-21T09:07:34.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:34 smithi105 ceph-mon[35993]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 999 MiB/s wr, 2.63k op/s 2024-03-21T09:07:35.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:34 smithi100 ceph-mon[36306]: pgmap v665: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 999 MiB/s wr, 2.63k op/s 2024-03-21T09:07:36.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:36 smithi087 ceph-mon[32385]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 763 MiB/s wr, 2.01k op/s 2024-03-21T09:07:36.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:36 smithi105 ceph-mon[35993]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 763 MiB/s wr, 2.01k op/s 2024-03-21T09:07:37.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:36 smithi100 ceph-mon[36306]: pgmap v666: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 763 MiB/s wr, 2.01k op/s 2024-03-21T09:07:38.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:38 smithi087 ceph-mon[32385]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 822 MiB/s wr, 2.47k op/s 2024-03-21T09:07:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:38 smithi105 ceph-mon[35993]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 822 MiB/s wr, 2.47k op/s 2024-03-21T09:07:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:38 smithi100 ceph-mon[36306]: pgmap v667: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 MiB/s rd, 822 MiB/s wr, 2.47k op/s 2024-03-21T09:07:40.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:40 smithi087 ceph-mon[32385]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 883 MiB/s wr, 3.23k op/s 2024-03-21T09:07:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:40 smithi105 ceph-mon[35993]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 883 MiB/s wr, 3.23k op/s 2024-03-21T09:07:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:40 smithi100 ceph-mon[36306]: pgmap v668: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 883 MiB/s wr, 3.23k op/s 2024-03-21T09:07:42.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:42 smithi087 ceph-mon[32385]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 720 MiB/s wr, 3.14k op/s 2024-03-21T09:07:42.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:42 smithi105 ceph-mon[35993]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 720 MiB/s wr, 3.14k op/s 2024-03-21T09:07:43.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:42 smithi100 ceph-mon[36306]: pgmap v669: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 720 MiB/s wr, 3.14k op/s 2024-03-21T09:07:44.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:44 smithi087 ceph-mon[32385]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 929 MiB/s wr, 4.48k op/s 2024-03-21T09:07:44.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:44 smithi105 ceph-mon[35993]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 929 MiB/s wr, 4.48k op/s 2024-03-21T09:07:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:44 smithi100 ceph-mon[36306]: pgmap v670: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 MiB/s rd, 929 MiB/s wr, 4.48k op/s 2024-03-21T09:07:46.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:46 smithi087 ceph-mon[32385]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 690 MiB/s wr, 3.71k op/s 2024-03-21T09:07:46.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:46 smithi105 ceph-mon[35993]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 690 MiB/s wr, 3.71k op/s 2024-03-21T09:07:46.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:46 smithi100 ceph-mon[36306]: pgmap v671: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 690 MiB/s wr, 3.71k op/s 2024-03-21T09:07:48.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:48 smithi100 ceph-mon[36306]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 772 MiB/s wr, 3.72k op/s 2024-03-21T09:07:48.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:48 smithi087 ceph-mon[32385]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 772 MiB/s wr, 3.72k op/s 2024-03-21T09:07:48.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:48 smithi105 ceph-mon[35993]: pgmap v672: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 772 MiB/s wr, 3.72k op/s 2024-03-21T09:07:50.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:50 smithi100 ceph-mon[36306]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 893 MiB/s wr, 3.67k op/s 2024-03-21T09:07:50.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:50 smithi087 ceph-mon[32385]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 893 MiB/s wr, 3.67k op/s 2024-03-21T09:07:50.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:50 smithi105 ceph-mon[35993]: pgmap v673: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 893 MiB/s wr, 3.67k op/s 2024-03-21T09:07:52.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:52 smithi087 ceph-mon[32385]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 759 MiB/s wr, 2.77k op/s 2024-03-21T09:07:52.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:52 smithi105 ceph-mon[35993]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 759 MiB/s wr, 2.77k op/s 2024-03-21T09:07:53.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:52 smithi100 ceph-mon[36306]: pgmap v674: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 759 MiB/s wr, 2.77k op/s 2024-03-21T09:07:54.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:54 smithi087 ceph-mon[32385]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1016 MiB/s wr, 3.53k op/s 2024-03-21T09:07:54.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:54 smithi105 ceph-mon[35993]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1016 MiB/s wr, 3.53k op/s 2024-03-21T09:07:55.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:54 smithi100 ceph-mon[36306]: pgmap v675: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 1016 MiB/s wr, 3.53k op/s 2024-03-21T09:07:56.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:56 smithi087 ceph-mon[32385]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 774 MiB/s wr, 2.21k op/s 2024-03-21T09:07:56.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:56 smithi105 ceph-mon[35993]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 774 MiB/s wr, 2.21k op/s 2024-03-21T09:07:57.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:56 smithi100 ceph-mon[36306]: pgmap v676: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 774 MiB/s wr, 2.21k op/s 2024-03-21T09:07:58.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:58 smithi087 ceph-mon[32385]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 836 MiB/s wr, 2.62k op/s 2024-03-21T09:07:58.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:58 smithi105 ceph-mon[35993]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 836 MiB/s wr, 2.62k op/s 2024-03-21T09:07:59.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:58 smithi100 ceph-mon[36306]: pgmap v677: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 20 MiB/s rd, 836 MiB/s wr, 2.62k op/s 2024-03-21T09:07:59.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:07:59 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:07:59.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:07:59 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:00.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:07:59 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:08:00.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:00 smithi100 ceph-mon[36306]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 892 MiB/s wr, 3.37k op/s 2024-03-21T09:08:00.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:00 smithi087 ceph-mon[32385]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 892 MiB/s wr, 3.37k op/s 2024-03-21T09:08:00.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:00 smithi105 ceph-mon[35993]: pgmap v678: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 892 MiB/s wr, 3.37k op/s 2024-03-21T09:08:02.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:02 smithi087 ceph-mon[32385]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 722 MiB/s wr, 3.21k op/s 2024-03-21T09:08:02.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:02 smithi105 ceph-mon[35993]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 722 MiB/s wr, 3.21k op/s 2024-03-21T09:08:03.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:02 smithi100 ceph-mon[36306]: pgmap v679: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 722 MiB/s wr, 3.21k op/s 2024-03-21T09:08:03.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:03.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:03.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:03.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:03 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:03 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:04.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:03 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:05.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:04 smithi105 ceph-mon[35993]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 862 MiB/s wr, 4.72k op/s 2024-03-21T09:08:05.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:05.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:04 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:05.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:04 smithi100 ceph-mon[36306]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 862 MiB/s wr, 4.72k op/s 2024-03-21T09:08:05.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:05.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:04 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:05.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:04 smithi087 ceph-mon[32385]: pgmap v680: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 26 MiB/s rd, 862 MiB/s wr, 4.72k op/s 2024-03-21T09:08:05.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:05.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:04 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:06.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:06.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:06.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:05 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:06.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:06.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:06.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:05 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:06.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:08:06.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:08:06.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:05 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:08:07.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:06 smithi105 ceph-mon[35993]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 597 MiB/s wr, 3.98k op/s 2024-03-21T09:08:07.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:06 smithi100 ceph-mon[36306]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 597 MiB/s wr, 3.98k op/s 2024-03-21T09:08:07.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:06 smithi087 ceph-mon[32385]: pgmap v681: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 21 MiB/s rd, 597 MiB/s wr, 3.98k op/s 2024-03-21T09:08:09.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:08 smithi105 ceph-mon[35993]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 636 MiB/s wr, 4.48k op/s 2024-03-21T09:08:09.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:08 smithi100 ceph-mon[36306]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 636 MiB/s wr, 4.48k op/s 2024-03-21T09:08:09.407 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:08 smithi087 ceph-mon[32385]: pgmap v682: 97 pgs: 97 active+clean; 10 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 25 MiB/s rd, 636 MiB/s wr, 4.48k op/s 2024-03-21T09:08:11.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:11 smithi100 ceph-mon[36306]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 708 MiB/s wr, 5.37k op/s 2024-03-21T09:08:11.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:11 smithi087 ceph-mon[32385]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 708 MiB/s wr, 5.37k op/s 2024-03-21T09:08:11.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:11 smithi105 ceph-mon[35993]: pgmap v683: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 MiB/s rd, 708 MiB/s wr, 5.37k op/s 2024-03-21T09:08:13.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:13 smithi100 ceph-mon[36306]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 574 MiB/s wr, 4.79k op/s 2024-03-21T09:08:13.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:13 smithi087 ceph-mon[32385]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 574 MiB/s wr, 4.79k op/s 2024-03-21T09:08:13.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:13 smithi105 ceph-mon[35993]: pgmap v684: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 574 MiB/s wr, 4.79k op/s 2024-03-21T09:08:15.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:15 smithi087 ceph-mon[32385]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 721 MiB/s wr, 5.76k op/s 2024-03-21T09:08:15.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:15 smithi105 ceph-mon[35993]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 721 MiB/s wr, 5.76k op/s 2024-03-21T09:08:15.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:15 smithi100 ceph-mon[36306]: pgmap v685: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 65 MiB/s rd, 721 MiB/s wr, 5.76k op/s 2024-03-21T09:08:17.282 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:17 smithi105 ceph-mon[35993]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 523 MiB/s wr, 4.13k op/s 2024-03-21T09:08:17.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:17 smithi087 ceph-mon[32385]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 523 MiB/s wr, 4.13k op/s 2024-03-21T09:08:17.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:17 smithi100 ceph-mon[36306]: pgmap v686: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 58 MiB/s rd, 523 MiB/s wr, 4.13k op/s 2024-03-21T09:08:19.155 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:19 smithi105 ceph-mon[35993]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 584 MiB/s wr, 4.49k op/s 2024-03-21T09:08:19.316 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:19 smithi100 ceph-mon[36306]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 584 MiB/s wr, 4.49k op/s 2024-03-21T09:08:19.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:19 smithi087 ceph-mon[32385]: pgmap v687: 97 pgs: 97 active+clean; 10 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 584 MiB/s wr, 4.49k op/s 2024-03-21T09:08:21.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:21 smithi087 ceph-mon[32385]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 653 MiB/s wr, 4.76k op/s 2024-03-21T09:08:21.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:21 smithi105 ceph-mon[35993]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 653 MiB/s wr, 4.76k op/s 2024-03-21T09:08:21.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:21 smithi100 ceph-mon[36306]: pgmap v688: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 61 MiB/s rd, 653 MiB/s wr, 4.76k op/s 2024-03-21T09:08:23.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:23 smithi087 ceph-mon[32385]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 498 MiB/s wr, 3.39k op/s 2024-03-21T09:08:23.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:23 smithi105 ceph-mon[35993]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 498 MiB/s wr, 3.39k op/s 2024-03-21T09:08:23.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:23 smithi100 ceph-mon[36306]: pgmap v689: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 498 MiB/s wr, 3.39k op/s 2024-03-21T09:08:25.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:25 smithi087 ceph-mon[32385]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 776 MiB/s wr, 4.56k op/s 2024-03-21T09:08:25.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:25 smithi105 ceph-mon[35993]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 776 MiB/s wr, 4.56k op/s 2024-03-21T09:08:25.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:25 smithi100 ceph-mon[36306]: pgmap v690: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 47 MiB/s rd, 776 MiB/s wr, 4.56k op/s 2024-03-21T09:08:25.814 INFO:tasks.workunit.client.0.smithi087.stdout:stopping iogen 2024-03-21T09:08:25.814 INFO:tasks.workunit.client.0.smithi087.stderr:+ echo 'stopping iogen' 2024-03-21T09:08:25.814 INFO:tasks.workunit.client.0.smithi087.stderr:+ ./iogen -k 2024-03-21T09:08:26.408 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:26 smithi087 ceph-mon[32385]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 607 MiB/s wr, 3.40k op/s 2024-03-21T09:08:26.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:26 smithi105 ceph-mon[35993]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 607 MiB/s wr, 3.40k op/s 2024-03-21T09:08:26.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:26 smithi100 ceph-mon[36306]: pgmap v691: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 607 MiB/s wr, 3.40k op/s 2024-03-21T09:08:28.906 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:28 smithi087 ceph-mon[32385]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 686 MiB/s wr, 3.69k op/s 2024-03-21T09:08:28.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:28 smithi105 ceph-mon[35993]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 686 MiB/s wr, 3.69k op/s 2024-03-21T09:08:29.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:28 smithi100 ceph-mon[36306]: pgmap v692: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 686 MiB/s wr, 3.69k op/s 2024-03-21T09:08:30.911 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:30 smithi087 ceph-mon[32385]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 829 MiB/s wr, 3.73k op/s 2024-03-21T09:08:30.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:30 smithi105 ceph-mon[35993]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 829 MiB/s wr, 3.73k op/s 2024-03-21T09:08:31.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:30 smithi100 ceph-mon[36306]: pgmap v693: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 829 MiB/s wr, 3.73k op/s 2024-03-21T09:08:32.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:32 smithi087 ceph-mon[32385]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 711 MiB/s wr, 2.93k op/s 2024-03-21T09:08:32.964 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:32 smithi100 ceph-mon[36306]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 711 MiB/s wr, 2.93k op/s 2024-03-21T09:08:32.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:32 smithi105 ceph-mon[35993]: pgmap v694: 97 pgs: 97 active+clean; 10 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 711 MiB/s wr, 2.93k op/s 2024-03-21T09:08:34.913 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:34 smithi087 ceph-mon[32385]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 968 MiB/s wr, 4.37k op/s 2024-03-21T09:08:34.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:34 smithi105 ceph-mon[35993]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 968 MiB/s wr, 4.37k op/s 2024-03-21T09:08:35.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:34 smithi100 ceph-mon[36306]: pgmap v695: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 968 MiB/s wr, 4.37k op/s 2024-03-21T09:08:36.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:36 smithi087 ceph-mon[32385]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 682 MiB/s wr, 2.92k op/s 2024-03-21T09:08:36.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:36 smithi105 ceph-mon[35993]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 682 MiB/s wr, 2.92k op/s 2024-03-21T09:08:37.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:36 smithi100 ceph-mon[36306]: pgmap v696: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 682 MiB/s wr, 2.92k op/s 2024-03-21T09:08:38.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:38 smithi087 ceph-mon[32385]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 748 MiB/s wr, 2.96k op/s 2024-03-21T09:08:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:38 smithi105 ceph-mon[35993]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 748 MiB/s wr, 2.96k op/s 2024-03-21T09:08:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:38 smithi100 ceph-mon[36306]: pgmap v697: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 748 MiB/s wr, 2.96k op/s 2024-03-21T09:08:40.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:40 smithi087 ceph-mon[32385]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 873 MiB/s wr, 3.26k op/s 2024-03-21T09:08:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:40 smithi105 ceph-mon[35993]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 873 MiB/s wr, 3.26k op/s 2024-03-21T09:08:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:40 smithi100 ceph-mon[36306]: pgmap v698: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 873 MiB/s wr, 3.26k op/s 2024-03-21T09:08:42.911 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:42 smithi087 ceph-mon[32385]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 693 MiB/s wr, 2.84k op/s 2024-03-21T09:08:42.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:42 smithi105 ceph-mon[35993]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 693 MiB/s wr, 2.84k op/s 2024-03-21T09:08:43.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:42 smithi100 ceph-mon[36306]: pgmap v699: 97 pgs: 97 active+clean; 10 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 693 MiB/s wr, 2.84k op/s 2024-03-21T09:08:44.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:44 smithi105 ceph-mon[35993]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 927 MiB/s wr, 3.34k op/s 2024-03-21T09:08:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:44 smithi100 ceph-mon[36306]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 927 MiB/s wr, 3.34k op/s 2024-03-21T09:08:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:44 smithi087 ceph-mon[32385]: pgmap v700: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 927 MiB/s wr, 3.34k op/s 2024-03-21T09:08:46.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:46 smithi105 ceph-mon[35993]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 730 MiB/s wr, 1.98k op/s 2024-03-21T09:08:47.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:46 smithi100 ceph-mon[36306]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 730 MiB/s wr, 1.98k op/s 2024-03-21T09:08:47.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:46 smithi087 ceph-mon[32385]: pgmap v701: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 730 MiB/s wr, 1.98k op/s 2024-03-21T09:08:48.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:48 smithi105 ceph-mon[35993]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 791 MiB/s wr, 1.96k op/s 2024-03-21T09:08:49.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:48 smithi100 ceph-mon[36306]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 791 MiB/s wr, 1.96k op/s 2024-03-21T09:08:49.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:48 smithi087 ceph-mon[32385]: pgmap v702: 97 pgs: 97 active+clean; 10 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 791 MiB/s wr, 1.96k op/s 2024-03-21T09:08:50.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:50 smithi105 ceph-mon[35993]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 928 MiB/s wr, 1.95k op/s 2024-03-21T09:08:51.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:50 smithi100 ceph-mon[36306]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 928 MiB/s wr, 1.95k op/s 2024-03-21T09:08:51.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:50 smithi087 ceph-mon[32385]: pgmap v703: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 928 MiB/s wr, 1.95k op/s 2024-03-21T09:08:53.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:52 smithi100 ceph-mon[36306]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 747 MiB/s wr, 1.31k op/s 2024-03-21T09:08:53.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:52 smithi087 ceph-mon[32385]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 747 MiB/s wr, 1.31k op/s 2024-03-21T09:08:53.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:52 smithi105 ceph-mon[35993]: pgmap v704: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 747 MiB/s wr, 1.31k op/s 2024-03-21T09:08:55.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:54 smithi100 ceph-mon[36306]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 958 MiB/s wr, 1.60k op/s 2024-03-21T09:08:55.158 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:54 smithi087 ceph-mon[32385]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 958 MiB/s wr, 1.60k op/s 2024-03-21T09:08:55.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:54 smithi105 ceph-mon[35993]: pgmap v705: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 958 MiB/s wr, 1.60k op/s 2024-03-21T09:08:57.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:56 smithi100 ceph-mon[36306]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 808 MiB/s wr, 1.14k op/s 2024-03-21T09:08:57.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:56 smithi087 ceph-mon[32385]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 808 MiB/s wr, 1.14k op/s 2024-03-21T09:08:57.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:56 smithi105 ceph-mon[35993]: pgmap v706: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 808 MiB/s wr, 1.14k op/s 2024-03-21T09:08:59.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:08:58 smithi087 ceph-mon[32385]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.13k op/s 2024-03-21T09:08:59.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:08:58 smithi105 ceph-mon[35993]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.13k op/s 2024-03-21T09:08:59.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:08:58 smithi100 ceph-mon[36306]: pgmap v707: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 835 MiB/s wr, 1.13k op/s 2024-03-21T09:09:01.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:00 smithi087 ceph-mon[32385]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 971 MiB/s wr, 1.39k op/s 2024-03-21T09:09:01.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:00 smithi105 ceph-mon[35993]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 971 MiB/s wr, 1.39k op/s 2024-03-21T09:09:01.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:00 smithi100 ceph-mon[36306]: pgmap v708: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 971 MiB/s wr, 1.39k op/s 2024-03-21T09:09:03.163 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:02 smithi087 ceph-mon[32385]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.22k op/s 2024-03-21T09:09:03.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:02 smithi105 ceph-mon[35993]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.22k op/s 2024-03-21T09:09:03.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:02 smithi100 ceph-mon[36306]: pgmap v709: 97 pgs: 97 active+clean; 10 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 778 MiB/s wr, 1.22k op/s 2024-03-21T09:09:05.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:04 smithi105 ceph-mon[35993]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1000 MiB/s wr, 1.60k op/s 2024-03-21T09:09:05.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:04 smithi100 ceph-mon[36306]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1000 MiB/s wr, 1.60k op/s 2024-03-21T09:09:05.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:04 smithi087 ceph-mon[32385]: pgmap v710: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1000 MiB/s wr, 1.60k op/s 2024-03-21T09:09:06.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:06 smithi105 ceph-mon[35993]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 MiB/s wr, 1.36k op/s 2024-03-21T09:09:06.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:06 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:09:06.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:06 smithi100 ceph-mon[36306]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 MiB/s wr, 1.36k op/s 2024-03-21T09:09:06.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:06 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:09:06.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:06 smithi087 ceph-mon[32385]: pgmap v711: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 869 MiB/s wr, 1.36k op/s 2024-03-21T09:09:06.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:06 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:09:08.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:08.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:07 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:08.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:08.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:07 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:08.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:08.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:07 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:09.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:09 smithi087 ceph-mon[32385]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 1.24k op/s 2024-03-21T09:09:09.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:09.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:09 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:09.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:09 smithi105 ceph-mon[35993]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 1.24k op/s 2024-03-21T09:09:09.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:09.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:09 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:09 smithi100 ceph-mon[36306]: pgmap v712: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 802 MiB/s wr, 1.24k op/s 2024-03-21T09:09:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:09.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:09 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:11.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:11 smithi087 ceph-mon[32385]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 969 MiB/s wr, 1.52k op/s 2024-03-21T09:09:11.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:09:11.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:09:11.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:11 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:11.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:11 smithi105 ceph-mon[35993]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 969 MiB/s wr, 1.52k op/s 2024-03-21T09:09:11.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:09:11.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:09:11.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:11 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:11.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:11 smithi100 ceph-mon[36306]: pgmap v713: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 969 MiB/s wr, 1.52k op/s 2024-03-21T09:09:11.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:09:11.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:09:11.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:11 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:09:13.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:13 smithi087 ceph-mon[32385]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 771 MiB/s wr, 1.25k op/s 2024-03-21T09:09:13.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:13 smithi105 ceph-mon[35993]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 771 MiB/s wr, 1.25k op/s 2024-03-21T09:09:13.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:13 smithi100 ceph-mon[36306]: pgmap v714: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 771 MiB/s wr, 1.25k op/s 2024-03-21T09:09:15.406 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:15 smithi087 ceph-mon[32385]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 987 MiB/s wr, 1.65k op/s 2024-03-21T09:09:15.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:15 smithi105 ceph-mon[35993]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 987 MiB/s wr, 1.65k op/s 2024-03-21T09:09:15.516 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:15 smithi100 ceph-mon[36306]: pgmap v715: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 987 MiB/s wr, 1.65k op/s 2024-03-21T09:09:17.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:17 smithi087 ceph-mon[32385]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 MiB/s wr, 1.43k op/s 2024-03-21T09:09:17.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:17 smithi105 ceph-mon[35993]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 MiB/s wr, 1.43k op/s 2024-03-21T09:09:17.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:17 smithi100 ceph-mon[36306]: pgmap v716: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 843 MiB/s wr, 1.43k op/s 2024-03-21T09:09:19.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:19 smithi087 ceph-mon[32385]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 775 MiB/s wr, 1.38k op/s 2024-03-21T09:09:19.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:19 smithi105 ceph-mon[35993]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 775 MiB/s wr, 1.38k op/s 2024-03-21T09:09:19.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:19 smithi100 ceph-mon[36306]: pgmap v717: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 775 MiB/s wr, 1.38k op/s 2024-03-21T09:09:21.410 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:21 smithi087 ceph-mon[32385]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 974 MiB/s wr, 2.02k op/s 2024-03-21T09:09:21.481 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:21 smithi105 ceph-mon[35993]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 974 MiB/s wr, 2.02k op/s 2024-03-21T09:09:21.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:21 smithi100 ceph-mon[36306]: pgmap v718: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 974 MiB/s wr, 2.02k op/s 2024-03-21T09:09:23.409 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:23 smithi087 ceph-mon[32385]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 741 MiB/s wr, 1.76k op/s 2024-03-21T09:09:23.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:23 smithi105 ceph-mon[35993]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 741 MiB/s wr, 1.76k op/s 2024-03-21T09:09:23.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:23 smithi100 ceph-mon[36306]: pgmap v719: 97 pgs: 97 active+clean; 10 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 741 MiB/s wr, 1.76k op/s 2024-03-21T09:09:25.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:25 smithi087 ceph-mon[32385]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 951 MiB/s wr, 2.58k op/s 2024-03-21T09:09:25.481 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:25 smithi105 ceph-mon[35993]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 951 MiB/s wr, 2.58k op/s 2024-03-21T09:09:25.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:25 smithi100 ceph-mon[36306]: pgmap v720: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 951 MiB/s wr, 2.58k op/s 2024-03-21T09:09:27.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:27 smithi087 ceph-mon[32385]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 806 MiB/s wr, 2.44k op/s 2024-03-21T09:09:27.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:27 smithi105 ceph-mon[35993]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 806 MiB/s wr, 2.44k op/s 2024-03-21T09:09:27.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:27 smithi100 ceph-mon[36306]: pgmap v721: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 806 MiB/s wr, 2.44k op/s 2024-03-21T09:09:29.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:29 smithi087 ceph-mon[32385]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 741 MiB/s wr, 2.34k op/s 2024-03-21T09:09:29.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:29 smithi105 ceph-mon[35993]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 741 MiB/s wr, 2.34k op/s 2024-03-21T09:09:29.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:29 smithi100 ceph-mon[36306]: pgmap v722: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 741 MiB/s wr, 2.34k op/s 2024-03-21T09:09:31.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:31 smithi087 ceph-mon[32385]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 957 MiB/s wr, 2.78k op/s 2024-03-21T09:09:31.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:31 smithi105 ceph-mon[35993]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 957 MiB/s wr, 2.78k op/s 2024-03-21T09:09:31.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:31 smithi100 ceph-mon[36306]: pgmap v723: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 957 MiB/s wr, 2.78k op/s 2024-03-21T09:09:33.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:33 smithi105 ceph-mon[35993]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 MiB/s wr, 2.12k op/s 2024-03-21T09:09:33.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:33 smithi100 ceph-mon[36306]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 MiB/s wr, 2.12k op/s 2024-03-21T09:09:33.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:33 smithi087 ceph-mon[32385]: pgmap v724: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 744 MiB/s wr, 2.12k op/s 2024-03-21T09:09:34.481 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:34 smithi105 ceph-mon[35993]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 975 MiB/s wr, 2.43k op/s 2024-03-21T09:09:34.515 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:34 smithi100 ceph-mon[36306]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 975 MiB/s wr, 2.43k op/s 2024-03-21T09:09:34.655 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:34 smithi087 ceph-mon[32385]: pgmap v725: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 975 MiB/s wr, 2.43k op/s 2024-03-21T09:09:36.910 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:36 smithi087 ceph-mon[32385]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 822 MiB/s wr, 1.75k op/s 2024-03-21T09:09:36.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:36 smithi105 ceph-mon[35993]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 822 MiB/s wr, 1.75k op/s 2024-03-21T09:09:37.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:36 smithi100 ceph-mon[36306]: pgmap v726: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 822 MiB/s wr, 1.75k op/s 2024-03-21T09:09:38.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:38 smithi087 ceph-mon[32385]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 MiB/s wr, 1.55k op/s 2024-03-21T09:09:38.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:38 smithi105 ceph-mon[35993]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 MiB/s wr, 1.55k op/s 2024-03-21T09:09:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:38 smithi100 ceph-mon[36306]: pgmap v727: 97 pgs: 97 active+clean; 10 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 778 MiB/s wr, 1.55k op/s 2024-03-21T09:09:40.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:40 smithi087 ceph-mon[32385]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 991 MiB/s wr, 1.98k op/s 2024-03-21T09:09:40.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:40 smithi105 ceph-mon[35993]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 991 MiB/s wr, 1.98k op/s 2024-03-21T09:09:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:40 smithi100 ceph-mon[36306]: pgmap v728: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 991 MiB/s wr, 1.98k op/s 2024-03-21T09:09:43.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:42 smithi087 ceph-mon[32385]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 773 MiB/s wr, 1.53k op/s 2024-03-21T09:09:43.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:42 smithi105 ceph-mon[35993]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 773 MiB/s wr, 1.53k op/s 2024-03-21T09:09:43.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:42 smithi100 ceph-mon[36306]: pgmap v729: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 773 MiB/s wr, 1.53k op/s 2024-03-21T09:09:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:44 smithi087 ceph-mon[32385]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 974 MiB/s wr, 2.04k op/s 2024-03-21T09:09:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:44 smithi105 ceph-mon[35993]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 974 MiB/s wr, 2.04k op/s 2024-03-21T09:09:45.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:44 smithi100 ceph-mon[36306]: pgmap v730: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 974 MiB/s wr, 2.04k op/s 2024-03-21T09:09:47.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:46 smithi105 ceph-mon[35993]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 804 MiB/s wr, 1.69k op/s 2024-03-21T09:09:47.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:46 smithi100 ceph-mon[36306]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 804 MiB/s wr, 1.69k op/s 2024-03-21T09:09:47.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:46 smithi087 ceph-mon[32385]: pgmap v731: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 804 MiB/s wr, 1.69k op/s 2024-03-21T09:09:49.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:48 smithi105 ceph-mon[35993]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 786 MiB/s wr, 1.66k op/s 2024-03-21T09:09:49.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:48 smithi100 ceph-mon[36306]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 786 MiB/s wr, 1.66k op/s 2024-03-21T09:09:49.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:48 smithi087 ceph-mon[32385]: pgmap v732: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 786 MiB/s wr, 1.66k op/s 2024-03-21T09:09:51.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:50 smithi105 ceph-mon[35993]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 979 MiB/s wr, 2.24k op/s 2024-03-21T09:09:51.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:50 smithi100 ceph-mon[36306]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 979 MiB/s wr, 2.24k op/s 2024-03-21T09:09:51.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:50 smithi087 ceph-mon[32385]: pgmap v733: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 979 MiB/s wr, 2.24k op/s 2024-03-21T09:09:53.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:52 smithi105 ceph-mon[35993]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 746 MiB/s wr, 1.79k op/s 2024-03-21T09:09:53.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:52 smithi100 ceph-mon[36306]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 746 MiB/s wr, 1.79k op/s 2024-03-21T09:09:53.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:52 smithi087 ceph-mon[32385]: pgmap v734: 97 pgs: 97 active+clean; 10 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 932 KiB/s rd, 746 MiB/s wr, 1.79k op/s 2024-03-21T09:09:55.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:54 smithi105 ceph-mon[35993]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 757 MiB/s wr, 1.76k op/s 2024-03-21T09:09:55.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:54 smithi100 ceph-mon[36306]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 757 MiB/s wr, 1.76k op/s 2024-03-21T09:09:55.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:54 smithi087 ceph-mon[32385]: pgmap v735: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 757 MiB/s wr, 1.76k op/s 2024-03-21T09:09:57.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:56 smithi087 ceph-mon[32385]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 524 MiB/s wr, 1.20k op/s 2024-03-21T09:09:57.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:56 smithi105 ceph-mon[35993]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 524 MiB/s wr, 1.20k op/s 2024-03-21T09:09:57.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:56 smithi100 ceph-mon[36306]: pgmap v736: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 524 MiB/s wr, 1.20k op/s 2024-03-21T09:09:59.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:09:58 smithi105 ceph-mon[35993]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 437 MiB/s wr, 1.09k op/s 2024-03-21T09:09:59.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:09:58 smithi100 ceph-mon[36306]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 437 MiB/s wr, 1.09k op/s 2024-03-21T09:09:59.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:09:58 smithi087 ceph-mon[32385]: pgmap v737: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 437 MiB/s wr, 1.09k op/s 2024-03-21T09:10:01.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:00 smithi105 ceph-mon[35993]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 391 MiB/s wr, 954 op/s 2024-03-21T09:10:01.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:00 smithi105 ceph-mon[35993]: overall HEALTH_OK 2024-03-21T09:10:01.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:00 smithi100 ceph-mon[36306]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 391 MiB/s wr, 954 op/s 2024-03-21T09:10:01.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:00 smithi100 ceph-mon[36306]: overall HEALTH_OK 2024-03-21T09:10:01.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:00 smithi087 ceph-mon[32385]: pgmap v738: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 391 MiB/s wr, 954 op/s 2024-03-21T09:10:01.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:00 smithi087 ceph-mon[32385]: overall HEALTH_OK 2024-03-21T09:10:03.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:02 smithi105 ceph-mon[35993]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 139 MiB/s wr, 251 op/s 2024-03-21T09:10:03.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:02 smithi100 ceph-mon[36306]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 139 MiB/s wr, 251 op/s 2024-03-21T09:10:03.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:02 smithi087 ceph-mon[32385]: pgmap v739: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 139 MiB/s wr, 251 op/s 2024-03-21T09:10:05.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:04 smithi105 ceph-mon[35993]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 133 MiB/s wr, 184 op/s 2024-03-21T09:10:05.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:04 smithi100 ceph-mon[36306]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 133 MiB/s wr, 184 op/s 2024-03-21T09:10:05.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:04 smithi087 ceph-mon[32385]: pgmap v740: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 133 MiB/s wr, 184 op/s 2024-03-21T09:10:07.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:06 smithi105 ceph-mon[35993]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 80 MiB/s wr, 101 op/s 2024-03-21T09:10:07.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:06 smithi100 ceph-mon[36306]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 80 MiB/s wr, 101 op/s 2024-03-21T09:10:07.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:06 smithi087 ceph-mon[32385]: pgmap v741: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 80 MiB/s wr, 101 op/s 2024-03-21T09:10:09.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:08 smithi105 ceph-mon[35993]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 79 MiB/s wr, 98 op/s 2024-03-21T09:10:09.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:08 smithi100 ceph-mon[36306]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 79 MiB/s wr, 98 op/s 2024-03-21T09:10:09.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:08 smithi087 ceph-mon[32385]: pgmap v742: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 79 MiB/s wr, 98 op/s 2024-03-21T09:10:11.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:10 smithi105 ceph-mon[35993]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 93 MiB/s wr, 114 op/s 2024-03-21T09:10:11.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:10 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:10:11.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:10 smithi100 ceph-mon[36306]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 93 MiB/s wr, 114 op/s 2024-03-21T09:10:11.265 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:10 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:10:11.404 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:10 smithi087 ceph-mon[32385]: pgmap v743: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 93 MiB/s wr, 114 op/s 2024-03-21T09:10:11.405 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:10 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:10:12.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:12 smithi087 ceph-mon[32385]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 71 MiB/s wr, 96 op/s 2024-03-21T09:10:12.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:12.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:12.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:10:12.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:10:12.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:12.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:12 smithi105 ceph-mon[35993]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 71 MiB/s wr, 96 op/s 2024-03-21T09:10:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:10:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:10:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:12 smithi100 ceph-mon[36306]: pgmap v744: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 71 MiB/s wr, 96 op/s 2024-03-21T09:10:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:10:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:10:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:10:14.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:14 smithi087 ceph-mon[32385]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 82 MiB/s wr, 104 op/s 2024-03-21T09:10:14.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:14 smithi105 ceph-mon[35993]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 82 MiB/s wr, 104 op/s 2024-03-21T09:10:15.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:14 smithi100 ceph-mon[36306]: pgmap v745: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 82 MiB/s wr, 104 op/s 2024-03-21T09:10:16.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:16 smithi087 ceph-mon[32385]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 62 MiB/s wr, 83 op/s 2024-03-21T09:10:16.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:16 smithi105 ceph-mon[35993]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 62 MiB/s wr, 83 op/s 2024-03-21T09:10:17.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:16 smithi100 ceph-mon[36306]: pgmap v746: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 62 MiB/s wr, 83 op/s 2024-03-21T09:10:18.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:18 smithi087 ceph-mon[32385]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 61 MiB/s wr, 80 op/s 2024-03-21T09:10:18.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:18 smithi105 ceph-mon[35993]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 61 MiB/s wr, 80 op/s 2024-03-21T09:10:19.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:18 smithi100 ceph-mon[36306]: pgmap v747: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 61 MiB/s wr, 80 op/s 2024-03-21T09:10:20.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:20 smithi087 ceph-mon[32385]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 82 MiB/s wr, 131 op/s 2024-03-21T09:10:20.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:20 smithi105 ceph-mon[35993]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 82 MiB/s wr, 131 op/s 2024-03-21T09:10:21.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:20 smithi100 ceph-mon[36306]: pgmap v748: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 82 MiB/s wr, 131 op/s 2024-03-21T09:10:22.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:22 smithi087 ceph-mon[32385]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 64 MiB/s wr, 162 op/s 2024-03-21T09:10:22.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:22 smithi105 ceph-mon[35993]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 64 MiB/s wr, 162 op/s 2024-03-21T09:10:23.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:22 smithi100 ceph-mon[36306]: pgmap v749: 97 pgs: 97 active+clean; 10 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 64 MiB/s wr, 162 op/s 2024-03-21T09:10:24.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:24 smithi087 ceph-mon[32385]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 83 MiB/s wr, 430 op/s 2024-03-21T09:10:24.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:24 smithi105 ceph-mon[35993]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 83 MiB/s wr, 430 op/s 2024-03-21T09:10:25.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:24 smithi100 ceph-mon[36306]: pgmap v750: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 83 MiB/s wr, 430 op/s 2024-03-21T09:10:26.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:26 smithi087 ceph-mon[32385]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 74 MiB/s wr, 488 op/s 2024-03-21T09:10:26.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:26 smithi105 ceph-mon[35993]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 74 MiB/s wr, 488 op/s 2024-03-21T09:10:27.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:26 smithi100 ceph-mon[36306]: pgmap v751: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 74 MiB/s wr, 488 op/s 2024-03-21T09:10:28.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:28 smithi087 ceph-mon[32385]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 69 MiB/s wr, 483 op/s 2024-03-21T09:10:28.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:28 smithi105 ceph-mon[35993]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 69 MiB/s wr, 483 op/s 2024-03-21T09:10:29.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:28 smithi100 ceph-mon[36306]: pgmap v752: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 69 MiB/s wr, 483 op/s 2024-03-21T09:10:30.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:30 smithi087 ceph-mon[32385]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 63 MiB/s wr, 473 op/s 2024-03-21T09:10:30.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:30 smithi105 ceph-mon[35993]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 63 MiB/s wr, 473 op/s 2024-03-21T09:10:31.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:30 smithi100 ceph-mon[36306]: pgmap v753: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 63 MiB/s wr, 473 op/s 2024-03-21T09:10:32.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:32 smithi087 ceph-mon[32385]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 36 MiB/s wr, 418 op/s 2024-03-21T09:10:32.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:32 smithi105 ceph-mon[35993]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 36 MiB/s wr, 418 op/s 2024-03-21T09:10:33.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:32 smithi100 ceph-mon[36306]: pgmap v754: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 36 MiB/s wr, 418 op/s 2024-03-21T09:10:34.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:34 smithi087 ceph-mon[32385]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 32 MiB/s wr, 370 op/s 2024-03-21T09:10:34.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:34 smithi105 ceph-mon[35993]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 32 MiB/s wr, 370 op/s 2024-03-21T09:10:35.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:34 smithi100 ceph-mon[36306]: pgmap v755: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 32 MiB/s wr, 370 op/s 2024-03-21T09:10:36.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:36 smithi087 ceph-mon[32385]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 95 op/s 2024-03-21T09:10:36.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:36 smithi105 ceph-mon[35993]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 95 op/s 2024-03-21T09:10:37.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:36 smithi100 ceph-mon[36306]: pgmap v756: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 95 op/s 2024-03-21T09:10:38.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:38 smithi087 ceph-mon[32385]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 24 op/s 2024-03-21T09:10:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:38 smithi105 ceph-mon[35993]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 24 op/s 2024-03-21T09:10:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:38 smithi100 ceph-mon[36306]: pgmap v757: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 24 op/s 2024-03-21T09:10:40.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:40 smithi087 ceph-mon[32385]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 28 op/s 2024-03-21T09:10:40.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:40 smithi105 ceph-mon[35993]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 28 op/s 2024-03-21T09:10:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:40 smithi100 ceph-mon[36306]: pgmap v758: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 28 op/s 2024-03-21T09:10:42.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:42 smithi087 ceph-mon[32385]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 25 op/s 2024-03-21T09:10:42.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:42 smithi105 ceph-mon[35993]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 25 op/s 2024-03-21T09:10:43.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:42 smithi100 ceph-mon[36306]: pgmap v759: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.4 MiB/s wr, 25 op/s 2024-03-21T09:10:44.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:44 smithi087 ceph-mon[32385]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-03-21T09:10:44.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:44 smithi105 ceph-mon[35993]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-03-21T09:10:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:44 smithi100 ceph-mon[36306]: pgmap v760: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.7 MiB/s wr, 27 op/s 2024-03-21T09:10:46.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:46 smithi087 ceph-mon[32385]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 21 op/s 2024-03-21T09:10:46.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:46 smithi105 ceph-mon[35993]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 21 op/s 2024-03-21T09:10:47.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:46 smithi100 ceph-mon[36306]: pgmap v761: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.9 MiB/s wr, 21 op/s 2024-03-21T09:10:48.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:48 smithi087 ceph-mon[32385]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-03-21T09:10:48.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:48 smithi105 ceph-mon[35993]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-03-21T09:10:49.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:48 smithi100 ceph-mon[36306]: pgmap v762: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.7 MiB/s wr, 29 op/s 2024-03-21T09:10:50.907 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:50 smithi087 ceph-mon[32385]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 33 MiB/s wr, 78 op/s 2024-03-21T09:10:50.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:50 smithi105 ceph-mon[35993]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 33 MiB/s wr, 78 op/s 2024-03-21T09:10:51.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:50 smithi100 ceph-mon[36306]: pgmap v763: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 33 MiB/s wr, 78 op/s 2024-03-21T09:10:52.427 INFO:tasks.workunit.client.0.smithi087.stdout:OK 2024-03-21T09:10:52.428 INFO:tasks.workunit.client.0.smithi087.stderr:+ echo OK 2024-03-21T09:10:52.707 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:52 smithi087 ceph-mon[32385]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 35 MiB/s wr, 78 op/s 2024-03-21T09:10:52.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:52 smithi105 ceph-mon[35993]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 35 MiB/s wr, 78 op/s 2024-03-21T09:10:53.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:52 smithi100 ceph-mon[36306]: pgmap v764: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 35 MiB/s wr, 78 op/s 2024-03-21T09:10:54.496 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-21T09:10:54.496 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-21T09:10:54.803 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:54 smithi087 ceph-mon[32385]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 69 MiB/s wr, 527 op/s 2024-03-21T09:10:54.813 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-03-21T09:10:54.813 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-21T09:10:54.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:54 smithi105 ceph-mon[35993]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 69 MiB/s wr, 527 op/s 2024-03-21T09:10:55.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:54 smithi100 ceph-mon[36306]: pgmap v765: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 69 MiB/s wr, 527 op/s 2024-03-21T09:10:55.703 DEBUG:teuthology.parallel:result is None 2024-03-21T09:10:55.703 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T09:10:55.740 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-21T09:10:55.740 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-03-21T09:10:55.752 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:10:56.387 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 -- 172.21.15.87:0/4179667291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401028a0 msgr2=0x7f5e40102d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 --2- 172.21.15.87:0/4179667291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401028a0 0x7f5e40102d20 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009900 tx=0x7f5e3002bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 -- 172.21.15.87:0/4179667291 shutdown_connections 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 --2- 172.21.15.87:0/4179667291 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e40103260 0x7f5e40109af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 --2- 172.21.15.87:0/4179667291 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401028a0 0x7f5e40102d20 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 --2- 172.21.15.87:0/4179667291 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e401016b0 0x7f5e40101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 -- 172.21.15.87:0/4179667291 >> 172.21.15.87:0/4179667291 conn(0x7f5e400fcc80 msgr2=0x7f5e400ff0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 -- 172.21.15.87:0/4179667291 shutdown_connections 2024-03-21T09:10:56.388 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.393+0000 7f5e44b32640 1 -- 172.21.15.87:0/4179667291 wait complete. 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 Processor -- start 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 -- start start 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 0x7f5e40199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e401028a0 0x7f5e4019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40103260 0x7f5e401a1170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4010d330 con 0x7f5e401016b0 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4010d1b0 con 0x7f5e40103260 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e44b32640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4010d4b0 con 0x7f5e401028a0 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 0x7f5e40199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.391 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 0x7f5e40199bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38594/0 (socket says 172.21.15.87:38594) 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 -- 172.21.15.87:0/2292729927 learned_addr learned my addr 172.21.15.87:0/2292729927 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3ed9d640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40103260 0x7f5e401a1170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3dd9b640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e401028a0 0x7f5e4019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 -- 172.21.15.87:0/2292729927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e401028a0 msgr2=0x7f5e4019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e401028a0 0x7f5e4019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 -- 172.21.15.87:0/2292729927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40103260 msgr2=0x7f5e401a1170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.395+0000 7f5e3e59c640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40103260 0x7f5e401a1170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e3e59c640 1 -- 172.21.15.87:0/2292729927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e401a17e0 con 0x7f5e401016b0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e3ed9d640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40103260 0x7f5e401a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e3e59c640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 0x7f5e40199bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e9c0 tx=0x7f5e3400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400cde0 con 0x7f5e401016b0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5e34010910 con 0x7f5e401016b0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e401a1ad0 con 0x7f5e401016b0 2024-03-21T09:10:56.392 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34018a00 con 0x7f5e401016b0 2024-03-21T09:10:56.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.396+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e401a1ff0 con 0x7f5e401016b0 2024-03-21T09:10:56.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.398+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5e3401f020 con 0x7f5e401016b0 2024-03-21T09:10:56.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.398+0000 7f5e2f7fe640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e280a2f20 0x7f5e280a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.399+0000 7f5e3dd9b640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e280a2f20 0x7f5e280a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.393 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.399+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5e340cf590 con 0x7f5e401016b0 2024-03-21T09:10:56.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.397+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e401a2220 con 0x7f5e401016b0 2024-03-21T09:10:56.395 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.401+0000 7f5e3dd9b640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e280a2f20 0x7f5e280a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002c7b0 tx=0x7f5e30005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:56.405 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.411+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5e3409c3f0 con 0x7f5e401016b0 2024-03-21T09:10:56.583 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.589+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5e401a2890 con 0x7f5e401016b0 2024-03-21T09:10:56.585 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.591+0000 7f5e2f7fe640 1 -- 172.21.15.87:0/2292729927 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7f5e3409bcf0 con 0x7f5e401016b0 2024-03-21T09:10:56.585 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T09:10:56.586 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:10:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e280a2f20 msgr2=0x7f5e280a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e280a2f20 0x7f5e280a53e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f5e3002c7b0 tx=0x7f5e30005fd0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 msgr2=0x7f5e40199bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.587 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 0x7f5e40199bc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e9c0 tx=0x7f5e3400ee80 comp rx=0 tx=0).stop 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 shutdown_connections 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5e280a2f20 0x7f5e280a53e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5e40103260 0x7f5e401a1170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5e401028a0 0x7f5e4019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 --2- 172.21.15.87:0/2292729927 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5e401016b0 0x7f5e40199bc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 >> 172.21.15.87:0/2292729927 conn(0x7f5e400fcc80 msgr2=0x7f5e400738d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 shutdown_connections 2024-03-21T09:10:56.588 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.593+0000 7f5e44b32640 1 -- 172.21.15.87:0/2292729927 wait complete. 2024-03-21T09:10:56.599 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:55:04.486296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:55:05.756309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T09:10:56.600 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:10:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.710+0000 7f29df3d4640 1 -- 172.21.15.87:0/1612030772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 msgr2=0x7f29d80ffb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.711 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.710+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1612030772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d80ffb90 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009900 tx=0x7f29d402bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- 172.21.15.87:0/1612030772 shutdown_connections 2024-03-21T09:10:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1612030772 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 0x7f29d8109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1612030772 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d80ffb90 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1612030772 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29d8102ca0 0x7f29d81030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.712 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- 172.21.15.87:0/1612030772 >> 172.21.15.87:0/1612030772 conn(0x7f29d80fb2d0 msgr2=0x7f29d80fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- 172.21.15.87:0/1612030772 shutdown_connections 2024-03-21T09:10:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- 172.21.15.87:0/1612030772 wait complete. 2024-03-21T09:10:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 Processor -- start 2024-03-21T09:10:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- start start 2024-03-21T09:10:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d8199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.713 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 0x7f29d819a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29d8102ca0 0x7f29d81a1190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d810d350 con 0x7f29d80ff710 2024-03-21T09:10:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d810d1d0 con 0x7f29d81000d0 2024-03-21T09:10:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29df3d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29d810d4d0 con 0x7f29d8102ca0 2024-03-21T09:10:56.714 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29dd170640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d8199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.715 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29dd170640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d8199c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38610/0 (socket says 172.21.15.87:38610) 2024-03-21T09:10:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29dd170640 1 -- 172.21.15.87:0/1189082329 learned_addr learned my addr 172.21.15.87:0/1189082329 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:10:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.711+0000 7f29dd971640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29d8102ca0 0x7f29d81a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dc96f640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 0x7f29d819a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.716 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dd170640 1 -- 172.21.15.87:0/1189082329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29d8102ca0 msgr2=0x7f29d81a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dd170640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29d8102ca0 0x7f29d81a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dd170640 1 -- 172.21.15.87:0/1189082329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 msgr2=0x7f29d819a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dd170640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 0x7f29d819a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dd170640 1 -- 172.21.15.87:0/1189082329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29d81a1830 con 0x7f29d80ff710 2024-03-21T09:10:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dc96f640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 0x7f29d819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:10:56.717 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29dd170640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d8199c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9c0 tx=0x7f29c800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:56.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c800cd80 con 0x7f29d80ff710 2024-03-21T09:10:56.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29d81a1b20 con 0x7f29d80ff710 2024-03-21T09:10:56.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f29c80035f0 con 0x7f29d80ff710 2024-03-21T09:10:56.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29c8010690 con 0x7f29d80ff710 2024-03-21T09:10:56.718 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.712+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29d81a2070 con 0x7f29d80ff710 2024-03-21T09:10:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.713+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29d81a22a0 con 0x7f29d80ff710 2024-03-21T09:10:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.714+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f29c80037a0 con 0x7f29d80ff710 2024-03-21T09:10:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.715+0000 7f29ce7fc640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29c40a2f20 0x7f29c40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.715+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f29c80cf8d0 con 0x7f29d80ff710 2024-03-21T09:10:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.715+0000 7f29dc96f640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29c40a2f20 0x7f29c40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:56.719 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.715+0000 7f29dc96f640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29c40a2f20 0x7f29c40a53e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f29d4006010 tx=0x7f29d402c8a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:56.720 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.718+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f29c809c730 con 0x7f29d80ff710 2024-03-21T09:10:56.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.894+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f29d81a2470 con 0x7f29d80ff710 2024-03-21T09:10:56.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.894+0000 7f29ce7fc640 1 -- 172.21.15.87:0/1189082329 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f29c809c030 con 0x7f29d80ff710 2024-03-21T09:10:56.889 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29c40a2f20 msgr2=0x7f29c40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29c40a2f20 0x7f29c40a53e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f29d4006010 tx=0x7f29d402c8a0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 msgr2=0x7f29d8199c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d8199c20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f29c800e9c0 tx=0x7f29c800ee80 comp rx=0 tx=0).stop 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 shutdown_connections 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f29c40a2f20 0x7f29c40a53e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f29d8102ca0 0x7f29d81a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f29d81000d0 0x7f29d819a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 --2- 172.21.15.87:0/1189082329 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f29d80ff710 0x7f29d8199c20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:56.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 >> 172.21.15.87:0/1189082329 conn(0x7f29d80fb2d0 msgr2=0x7f29d8100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:56.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 shutdown_connections 2024-03-21T09:10:56.892 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:56.896+0000 7f29df3d4640 1 -- 172.21.15.87:0/1189082329 wait complete. 2024-03-21T09:10:56.900 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:56 smithi087 ceph-mon[32385]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 81 MiB/s wr, 749 op/s 2024-03-21T09:10:56.900 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:56 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2292729927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:10:56.901 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:10:56.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:56 smithi105 ceph-mon[35993]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 81 MiB/s wr, 749 op/s 2024-03-21T09:10:56.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:56 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2292729927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:10:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.010+0000 7fa8709f3640 1 -- 172.21.15.87:0/3813223601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1028c0 msgr2=0x7fa86c102d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.010+0000 7fa8709f3640 1 --2- 172.21.15.87:0/3813223601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1028c0 0x7fa86c102d40 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa860009900 tx=0x7fa86002bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- 172.21.15.87:0/3813223601 shutdown_connections 2024-03-21T09:10:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 --2- 172.21.15.87:0/3813223601 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa86c103280 0x7fa86c109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 --2- 172.21.15.87:0/3813223601 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1028c0 0x7fa86c102d40 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.009 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 --2- 172.21.15.87:0/3813223601 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa86c1016d0 0x7fa86c101af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- 172.21.15.87:0/3813223601 >> 172.21.15.87:0/3813223601 conn(0x7fa86c0fcc40 msgr2=0x7fa86c0ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- 172.21.15.87:0/3813223601 shutdown_connections 2024-03-21T09:10:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- 172.21.15.87:0/3813223601 wait complete. 2024-03-21T09:10:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 Processor -- start 2024-03-21T09:10:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- start start 2024-03-21T09:10:57.010 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 0x7fa86c197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa86c1028c0 0x7fa86c197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa86c103280 0x7fa86c19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c10d350 con 0x7fa86c1016d0 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c10d1d0 con 0x7fa86c103280 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.011+0000 7fa8709f3640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa86c10d4d0 con 0x7fa86c1028c0 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 0x7fa86c197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 0x7fa86c197a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38622/0 (socket says 172.21.15.87:38622) 2024-03-21T09:10:57.011 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 -- 172.21.15.87:0/1177133020 learned_addr learned my addr 172.21.15.87:0/1177133020 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa869d9b640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa86c1028c0 0x7fa86c197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86ad9d640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa86c103280 0x7fa86c19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 -- 172.21.15.87:0/1177133020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa86c1028c0 msgr2=0x7fa86c197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa86c1028c0 0x7fa86c197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 -- 172.21.15.87:0/1177133020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa86c103280 msgr2=0x7fa86c19efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa86c103280 0x7fa86c19efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 -- 172.21.15.87:0/1177133020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86c19f640 con 0x7fa86c1016d0 2024-03-21T09:10:57.012 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa86a59c640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 0x7fa86c197a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00eb70 tx=0x7fa85c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c003b70 con 0x7fa86c1016d0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa85c00ee70 con 0x7fa86c1016d0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86c19f930 con 0x7fa86c1016d0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85c010830 con 0x7fa86c1016d0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.012+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86c19fe00 con 0x7fa86c1016d0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.013+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa86c108a70 con 0x7fa86c1016d0 2024-03-21T09:10:57.013 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.014+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa85c004a40 con 0x7fa86c1016d0 2024-03-21T09:10:57.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.015+0000 7fa85b7fe640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa8540a2f20 0x7fa8540a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.015+0000 7fa869d9b640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa8540a2f20 0x7fa8540a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.015+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa85c0cfae0 con 0x7fa86c1016d0 2024-03-21T09:10:57.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.015+0000 7fa869d9b640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa8540a2f20 0x7fa8540a53e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa86002c7b0 tx=0x7fa860005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.015 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.019+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa85c09c940 con 0x7fa86c1016d0 2024-03-21T09:10:57.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:56 smithi100 ceph-mon[36306]: pgmap v766: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 MiB/s rd, 81 MiB/s wr, 749 op/s 2024-03-21T09:10:57.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:56 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2292729927' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:10:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.192+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa86c0622a0 con 0x7fa86c1016d0 2024-03-21T09:10:57.187 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.193+0000 7fa85b7fe640 1 -- 172.21.15.87:0/1177133020 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa85c09c240 con 0x7fa86c1016d0 2024-03-21T09:10:57.187 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-03-21T09:10:57.189 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa8540a2f20 msgr2=0x7fa8540a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa8540a2f20 0x7fa8540a53e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa86002c7b0 tx=0x7fa860005fd0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 msgr2=0x7fa86c197a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 0x7fa86c197a20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00eb70 tx=0x7fa85c00c6a0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 shutdown_connections 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa8540a2f20 0x7fa8540a53e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa86c103280 0x7fa86c19efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa86c1028c0 0x7fa86c197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 --2- 172.21.15.87:0/1177133020 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa86c1016d0 0x7fa86c197a20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 >> 172.21.15.87:0/1177133020 conn(0x7fa86c0fcc40 msgr2=0x7fa86c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 shutdown_connections 2024-03-21T09:10:57.190 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.195+0000 7fa8709f3640 1 -- 172.21.15.87:0/1177133020 wait complete. 2024-03-21T09:10:57.200 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:10:57.307 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- 172.21.15.87:0/1508323008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7968102a10 msgr2=0x7f7968102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- 172.21.15.87:0/1508323008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7968102a10 0x7f7968102e90 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f795c009930 tx=0x7f795c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- 172.21.15.87:0/1508323008 shutdown_connections 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- 172.21.15.87:0/1508323008 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f79681033d0 0x7f7968109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- 172.21.15.87:0/1508323008 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7968102a10 0x7f7968102e90 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- 172.21.15.87:0/1508323008 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79680ff9a0 0x7f79680ffdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- 172.21.15.87:0/1508323008 >> 172.21.15.87:0/1508323008 conn(0x7f79680fb2d0 msgr2=0x7f79680fd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- 172.21.15.87:0/1508323008 shutdown_connections 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- 172.21.15.87:0/1508323008 wait complete. 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 Processor -- start 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- start start 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 0x7f7968197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968102a10 0x7f7968197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.308 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79681033d0 0x7f796819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796810d350 con 0x7f79680ff9a0 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796810d1d0 con 0x7f7968102a10 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.311+0000 7f796eb77640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796810d4d0 con 0x7f79681033d0 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 0x7f7968197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 0x7f7968197a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:38650/0 (socket says 172.21.15.87:38650) 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 -- 172.21.15.87:0/1360589334 learned_addr learned my addr 172.21.15.87:0/1360589334 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796d114640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79681033d0 0x7f796819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f7967fff640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968102a10 0x7f7968197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 -- 172.21.15.87:0/1360589334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79681033d0 msgr2=0x7f796819efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79681033d0 0x7f796819efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 -- 172.21.15.87:0/1360589334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968102a10 msgr2=0x7f7968197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968102a10 0x7f7968197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.309 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 -- 172.21.15.87:0/1360589334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796819f620 con 0x7f79680ff9a0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796c913640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 0x7f7968197a00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f795800e9c0 tx=0x7f795800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800cd80 con 0x7f79680ff9a0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796819f910 con 0x7f79680ff9a0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f79580035f0 con 0x7f79680ff9a0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958010690 con 0x7f79680ff9a0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.312+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796819fde0 con 0x7f79680ff9a0 2024-03-21T09:10:57.310 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.313+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968108b40 con 0x7f79680ff9a0 2024-03-21T09:10:57.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.314+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f79580037a0 con 0x7f79680ff9a0 2024-03-21T09:10:57.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.315+0000 7f7965ffb640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f79500a2f20 0x7f79500a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.315+0000 7f7967fff640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f79500a2f20 0x7f79500a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.315+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f79580cf720 con 0x7f79680ff9a0 2024-03-21T09:10:57.311 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.315+0000 7f7967fff640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f79500a2f20 0x7f79500a53e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f795c000c00 tx=0x7f795c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.313 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.319+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f795809c580 con 0x7f79680ff9a0 2024-03-21T09:10:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.495+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79680622a0 con 0x7f79680ff9a0 2024-03-21T09:10:57.490 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.495+0000 7f7965ffb640 1 -- 172.21.15.87:0/1360589334 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f795809be80 con 0x7f79680ff9a0 2024-03-21T09:10:57.490 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-03-21T09:10:57.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f79500a2f20 msgr2=0x7f79500a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.492 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f79500a2f20 0x7f79500a53e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f795c000c00 tx=0x7f795c005fd0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 msgr2=0x7f7968197a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 0x7f7968197a00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f795800e9c0 tx=0x7f795800ee80 comp rx=0 tx=0).stop 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 shutdown_connections 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f79500a2f20 0x7f79500a53e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f79681033d0 0x7f796819efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7968102a10 0x7f7968197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 --2- 172.21.15.87:0/1360589334 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f79680ff9a0 0x7f7968197a00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 >> 172.21.15.87:0/1360589334 conn(0x7f79680fb2d0 msgr2=0x7f7968105d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 shutdown_connections 2024-03-21T09:10:57.493 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.498+0000 7f796eb77640 1 -- 172.21.15.87:0/1360589334 wait complete. 2024-03-21T09:10:57.503 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-03-21T09:10:57.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 -- 172.21.15.87:0/1501517673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 msgr2=0x7f64f4108a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 --2- 172.21.15.87:0/1501517673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f4108a00 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009900 tx=0x7f64e402bc30 comp rx=0 tx=0).stop 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 -- 172.21.15.87:0/1501517673 shutdown_connections 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 --2- 172.21.15.87:0/1501517673 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f4108a00 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 --2- 172.21.15.87:0/1501517673 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f41017b0 0x7f64f4101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 --2- 172.21.15.87:0/1501517673 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41005c0 0x7f64f41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 -- 172.21.15.87:0/1501517673 >> 172.21.15.87:0/1501517673 conn(0x7f64f40fbb30 msgr2=0x7f64f40fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 -- 172.21.15.87:0/1501517673 shutdown_connections 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 -- 172.21.15.87:0/1501517673 wait complete. 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.611+0000 7f64f989a640 1 Processor -- start 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 -- start start 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f41005c0 0x7f64f4197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41017b0 0x7f64f4197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.608 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f419eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f410d360 con 0x7f64f4102170 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f410d1e0 con 0x7f64f41005c0 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f410d4e0 con 0x7f64f41017b0 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f419eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f419eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:50732/0 (socket says 172.21.15.87:50732) 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f27fc640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41017b0 0x7f64f4197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 -- 172.21.15.87:0/619019988 learned_addr learned my addr 172.21.15.87:0/619019988 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f2ffd640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f41005c0 0x7f64f4197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 -- 172.21.15.87:0/619019988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41017b0 msgr2=0x7f64f4197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41017b0 0x7f64f4197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 -- 172.21.15.87:0/619019988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f41005c0 msgr2=0x7f64f4197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f41005c0 0x7f64f4197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 -- 172.21.15.87:0/619019988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f419f540 con 0x7f64f4102170 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f37fe640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f419eed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005ab0 tx=0x7f64e402c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.609 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4041070 con 0x7f64f4102170 2024-03-21T09:10:57.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f419f7d0 con 0x7f64f4102170 2024-03-21T09:10:57.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f64e4004450 con 0x7f64f4102170 2024-03-21T09:10:57.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.612+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4003e60 con 0x7f64f4102170 2024-03-21T09:10:57.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.613+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f419fcc0 con 0x7f64f4102170 2024-03-21T09:10:57.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.614+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f41078e0 con 0x7f64f4102170 2024-03-21T09:10:57.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.614+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f64e4044070 con 0x7f64f4102170 2024-03-21T09:10:57.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.615+0000 7f64f8898640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f64e00a2f20 0x7f64e00a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:10:57.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.615+0000 7f64f2ffd640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f64e00a2f20 0x7f64e00a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:10:57.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.615+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f64e40f0ff0 con 0x7f64f4102170 2024-03-21T09:10:57.611 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.615+0000 7f64f2ffd640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f64e00a2f20 0x7f64e00a53e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f64f4101610 tx=0x7f64ec009500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:10:57.614 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.619+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f64e40bd800 con 0x7f64f4102170 2024-03-21T09:10:57.835 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.840+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f64f40622a0 con 0x7f64f4102170 2024-03-21T09:10:57.843 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.849+0000 7f64f8898640 1 -- 172.21.15.87:0/619019988 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) v1 ==== 81+0+741841 (secure 0 0 0) 0x7f64e40c18a0 con 0x7f64f4102170 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "epoch": 20, 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "active_gid": 14152, 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "active_name": "x", 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "active_addrs": { 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-03-21T09:10:57.854 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:6800", 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1005139006 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v1", 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:6801", 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1005139006 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "active_addr": "172.21.15.87:6801/1005139006", 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "active_change": "2024-03-21T08:47:58.771601+0000", 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "active_mgr_features": 4540701547738038271, 2024-03-21T09:10:57.855 INFO:teuthology.orchestra.run.smithi087.stdout: "available": true, 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "standbys": [ 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "gid": 24109, 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "z", 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr_features": 4540701547738038271, 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "available_modules": [ 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "alerts", 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:57.856 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:57.857 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.858 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.859 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.860 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:57.861 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_destination": { 2024-03-21T09:10:57.862 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_destination", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email address to send alerts to", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_from_name": { 2024-03-21T09:10:57.863 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_from_name", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "Ceph", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email From: name", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.864 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_host": { 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_host", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP server", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_password": { 2024-03-21T09:10:57.865 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_password", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Password to authenticate with", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.866 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_port": { 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_port", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "465", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP port", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.867 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_sender": { 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_sender", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP envelope sender", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.868 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_ssl": { 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_ssl", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.869 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_user": { 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_user", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "User to authenticate as", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.870 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "balancer", 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "active": { 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "active", 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.871 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_time": { 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_time", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0000", 2024-03-21T09:10:57.872 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_weekday": { 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_weekday", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-03-21T09:10:57.873 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_max_iterations": { 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_max_iterations", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "25", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:57.874 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "250", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_metrics": { 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_metrics", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.875 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_step": { 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_step", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.5", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0.001", 2024-03-21T09:10:57.876 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "0.999", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "aggressiveness of optimization", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "end_time": { 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_time", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2359", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.877 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "end_weekday": { 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_weekday", 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.878 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.879 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.880 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.882 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.883 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "min_score": { 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_score", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:57.884 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "upmap", 2024-03-21T09:10:57.885 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "crush-compat", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "none", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "read", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap-read" 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Balancer mode", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_ids": { 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_ids", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.888 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_deviation": { 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_deviation", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_optimizations": { 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_optimizations", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.890 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm", 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_down_multiplier": { 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_down_multiplier", 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.891 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_refresh_rate": { 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_refresh_rate", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "20", 2024-03-21T09:10:57.892 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_starting_port": { 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_starting_port", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.893 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4721", 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_ptrace": { 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_ptrace", 2024-03-21T09:10:57.894 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_interval": { 2024-03-21T09:10:57.895 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_interval", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.896 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_memory_target_ratio": { 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.7", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm_log_destination": { 2024-03-21T09:10:57.897 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm_log_destination", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "file", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "file,syslog", 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "syslog" 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.898 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "cgroups_split": { 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cgroups_split", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.899 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "config_checks_enabled": { 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_checks_enabled", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.900 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "config_dashboard": { 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_dashboard", 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.901 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_alertmanager": { 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_alertmanager", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T09:10:57.902 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_base": { 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_base", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.903 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Container image name, without the tag", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_elasticsearch": { 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_elasticsearch", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.904 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "elasticsearch container image", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_grafana": { 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_grafana", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.905 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_haproxy": { 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_haproxy", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.906 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "HAproxy container image", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_agent": { 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_agent", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.907 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger agent container image", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_collector": { 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_collector", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.908 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger collector container image", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_query": { 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_query", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.909 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger query container image", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_keepalived": { 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_keepalived", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.910 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Keepalived container image", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_loki": { 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_loki", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.911 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Loki container image", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_node_exporter": { 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_node_exporter", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T09:10:57.912 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_nvmeof": { 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_nvmeof", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T09:10:57.913 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Nvme-of container image", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_prometheus": { 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_prometheus", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.914 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_promtail": { 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_promtail", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.915 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Promtail container image", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_snmp_gateway": { 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_snmp_gateway", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.916 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SNMP Gateway container image", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "container_init": { 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_init", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.917 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "daemon_cache_timeout": { 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "daemon_cache_timeout", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.918 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "default_cephadm_command_timeout": { 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.919 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "900", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "default_registry": { 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_registry", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.920 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "device_cache_timeout": { 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_cache_timeout", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.921 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1800", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache device inventory", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "device_enhanced_scan": { 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_enhanced_scan", 2024-03-21T09:10:57.922 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "facts_cache_timeout": { 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "facts_cache_timeout", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.923 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache host facts data", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "grafana_dashboards_path": { 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "grafana_dashboards_path", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.924 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "host_check_interval": { 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "host_check_interval", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.925 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to perform a host check", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "hw_monitoring": { 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hw_monitoring", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.926 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "inventory_list_all": { 2024-03-21T09:10:57.927 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "inventory_list_all", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:57.928 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.929 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "log_refresh_metadata": { 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_refresh_metadata", 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.930 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.931 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.932 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.933 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.934 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.935 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.936 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "*", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "max_count_per_host": { 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_count_per_host", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.937 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of daemons per service per host", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "max_osd_draining_count": { 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_osd_draining_count", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.938 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "migration_current": { 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "migration_current", 2024-03-21T09:10:57.939 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "internal - do not modify", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-03-21T09:10:57.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "root", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm-package", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "root" 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.941 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "oob_default_addr": { 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "oob_default_addr", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "169.254.1.1", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.942 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "prometheus_alerts_path": { 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus_alerts_path", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.943 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_insecure": { 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_insecure", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.944 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_password": { 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_password", 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.945 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_url": { 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_url", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.946 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_username": { 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_username", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.947 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "secure_monitoring_stack": { 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "secure_monitoring_stack", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.948 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "service_discovery_port": { 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "service_discovery_port", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8765", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.949 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "cephadm service discovery port", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "ssh_config_file": { 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssh_config_file", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.950 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "use_agent": { 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_agent", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.951 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "use_repo_digest": { 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_repo_digest", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.952 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_failed_host_check": { 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_failed_host_check", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.953 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_daemons": { 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_daemons", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.954 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_hosts": { 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_hosts", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.955 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crash", 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:57.956 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.957 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.958 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.959 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:57.960 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.961 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "retain_interval": { 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retain_interval", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "31536000", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.962 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_recent_interval": { 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_recent_interval", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1209600", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.963 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dashboard", 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:57.964 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.966 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.967 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_ENABLED": { 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.968 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.969 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.970 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.971 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.972 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.973 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.974 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.975 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.976 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.977 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T09:10:57.978 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T09:10:57.979 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T09:10:57.980 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_URL": { 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_URL", 2024-03-21T09:10:57.981 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.982 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T09:10:57.983 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.984 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.985 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.986 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.987 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "{}", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.988 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.989 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.990 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.991 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.992 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.993 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T09:10:57.994 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T09:10:57.995 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T09:10:57.996 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T09:10:57.997 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T09:10:57.998 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:57.999 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.001 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.002 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "45", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.003 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.004 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.005 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.006 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.007 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.008 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.009 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.010 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.011 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "cross_origin_url": { 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cross_origin_url", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.012 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "crt_file": { 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crt_file", 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.013 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "debug": { 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "debug", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.014 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable/disable debug options", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "jwt_token_ttl": { 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "jwt_token_ttl", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "28800", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.015 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.016 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.017 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.018 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.019 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.020 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.021 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "motd": { 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "motd", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.022 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "The message of the day", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect_resolve_ip_addr": { 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.023 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.024 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8080", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.025 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.026 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl_server_port": { 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl_server_port", 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8443", 2024-03-21T09:10:58.027 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "redirect", 2024-03-21T09:10:58.028 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect" 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.029 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.030 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "url_prefix": { 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url_prefix", 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.031 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_monitoring": { 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_monitoring", 2024-03-21T09:10:58.032 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "monitor device health metrics", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.033 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.034 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.035 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.036 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.037 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "mark_out_threshold": { 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mark_out_threshold", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.038 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2419200", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_name": { 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_name", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.039 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "device_health_metrics", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "retention_period": { 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retention_period", 2024-03-21T09:10:58.040 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15552000", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain device health metrics", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.041 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_frequency": { 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_frequency", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "self_heal": { 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "self_heal", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.043 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T09:10:58.044 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_threshold": { 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_threshold", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7257600", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T09:10:58.045 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "diskprediction_local", 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.046 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.047 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.048 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.049 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.050 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "predict_interval": { 2024-03-21T09:10:58.051 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predict_interval", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "predictor_model": { 2024-03-21T09:10:58.052 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predictor_model", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "prophetstor", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.053 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.054 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "influx", 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": false, 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "influxdb python module not found", 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "batch_size": { 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "batch_size", 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.055 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5000", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "database": { 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "database", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.056 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "ceph", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": { 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hostname", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.057 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server hostname", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.058 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "30", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "5", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.059 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.060 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.061 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.062 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.064 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "password": { 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "password", 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.065 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "password of InfluxDB server user", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "port": { 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "port", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8086", 2024-03-21T09:10:58.066 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server port", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "false", 2024-03-21T09:10:58.067 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "threads": { 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threads", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.068 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "32", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "username": { 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "username", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.069 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "username of InfluxDB server user", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "verify_ssl": { 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "verify_ssl", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.070 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "true", 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.071 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "insights", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.072 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.074 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.075 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.076 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "iostat", 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.077 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.078 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.079 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.080 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.081 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.082 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "k8sevents", 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_event_retention_days": { 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ceph_event_retention_days", 2024-03-21T09:10:58.083 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "config_check_secs": { 2024-03-21T09:10:58.084 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_check_secs", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "10", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.085 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.086 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.087 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.088 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.089 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.090 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "localpool", 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "failure_domain": { 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "failure_domain", 2024-03-21T09:10:58.091 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "host", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "failure domain for any created local pool", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.092 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.093 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.094 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.095 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.096 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.097 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "min_size": { 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_size", 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.098 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default min_size for any created local pool", 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "num_rep": { 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "num_rep", 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3", 2024-03-21T09:10:58.099 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default replica count for any created local pool", 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_num": { 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_num", 2024-03-21T09:10:58.100 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "128", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "prefix": { 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prefix", 2024-03-21T09:10:58.101 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name prefix for any created local pool", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.102 INFO:teuthology.orchestra.run.smithi087.stdout: "subtree": { 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "subtree", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "rack", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.103 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mds_autoscaler", 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.104 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.105 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.106 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.107 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.108 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.109 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirroring", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.110 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.111 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.112 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.113 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.114 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "nfs", 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.115 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.116 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.117 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.118 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.119 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.120 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "fail_fs": { 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "fail_fs", 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.121 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.122 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.123 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.124 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.125 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.126 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.127 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator": { 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-03-21T09:10:58.128 INFO:teuthology.orchestra.run.smithi087.stdout: "rook", 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "test_orchestrator" 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Orchestrator backend", 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_perf_query", 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.129 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.130 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.131 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.132 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.133 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.134 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_support", 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.135 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.136 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.137 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.138 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.139 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_autoscaler", 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.141 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.142 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.143 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.144 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.145 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "threshold": { 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threshold", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1.0", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "scaling threshold", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.148 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "progress", 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_pg_recovery_event": { 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_pg_recovery_event", 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.149 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-03-21T09:10:58.150 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.151 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.152 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.153 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.154 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.155 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "max_completed_events": { 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_completed_events", 2024-03-21T09:10:58.156 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "50", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "number of past completed events to remember", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.157 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.158 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long the module is going to sleep", 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus", 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "cache": { 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cache", 2024-03-21T09:10:58.159 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "exclude_perf_counters": { 2024-03-21T09:10:58.160 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "exclude_perf_counters", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.161 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.162 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.163 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.164 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.165 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.166 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools": { 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.167 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T09:10:58.169 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "300", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_interval": { 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_interval", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15.0", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.172 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "9283", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.173 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "stale_cache_strategy": { 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stale_cache_strategy", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "log", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.174 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "default", 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.175 INFO:teuthology.orchestra.run.smithi087.stdout: "default", 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "error" 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.176 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.177 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.178 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.179 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.181 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.182 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_snap_create": { 2024-03-21T09:10:58.183 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_snap_create", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "mirror_snapshot_schedule": { 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.185 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "trash_purge_schedule": { 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "trash_purge_schedule", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.186 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "restful", 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_auth": { 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_auth", 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.187 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.189 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.190 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.191 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.192 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.193 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.194 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.195 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.196 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rgw", 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.197 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.198 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.199 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.200 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.201 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.202 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rook", 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.203 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.204 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.205 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.206 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.207 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.208 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.209 INFO:teuthology.orchestra.run.smithi087.stdout: "storage_class": { 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "storage_class", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "local", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.210 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "selftest", 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.211 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.212 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.213 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.214 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.215 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "roption1": { 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption1", 2024-03-21T09:10:58.216 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "roption2": { 2024-03-21T09:10:58.217 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption2", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "xyz", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.218 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption1": { 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption1", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.219 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption2": { 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption2", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.220 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption3": { 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption3", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.221 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption4": { 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption4", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.222 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption5": { 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption5", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.223 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption6": { 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption6", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.224 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption7": { 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption7", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.225 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "42", 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "testkey": { 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testkey", 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.226 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "testlkey": { 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testlkey", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.227 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "testnewline": { 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testnewline", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.228 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.229 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_m_granularity": { 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_m_granularity", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.230 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "dump_on_update": { 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dump_on_update", 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.231 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "dump database to debug log on update", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.232 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.233 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.234 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.235 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.236 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stats", 2024-03-21T09:10:58.237 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.238 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.239 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.240 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.241 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.242 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.243 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.244 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.245 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.246 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "status", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.247 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.248 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.249 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.250 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.251 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.252 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telegraf", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "address": { 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "address", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.253 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.254 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.255 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.256 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.257 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.258 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.259 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telemetry", 2024-03-21T09:10:58.260 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_basic": { 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_basic", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T09:10:58.261 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_crash": { 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_crash", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.262 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_device": { 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_device", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.263 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_ident": { 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_ident", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.264 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_perf": { 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_perf", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.265 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "contact": { 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "contact", 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.266 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "description": { 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "description", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.267 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "device_url": { 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_url", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T09:10:58.268 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.269 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.270 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "24", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "8", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "last_opt_revision": { 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "last_opt_revision", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.271 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard": { 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.272 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard_description": { 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard_description", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.273 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.274 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.275 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.276 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.277 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.278 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "organization": { 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "organization", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.279 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "proxy": { 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "proxy", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.280 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "url": { 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.281 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.282 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "test_orchestrator", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.283 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.284 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.285 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.286 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.287 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.288 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.289 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.290 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.291 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.292 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.293 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_clones": { 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_clones", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.294 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "periodic_async_work": { 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "periodic_async_work", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Periodically check for async work", 2024-03-21T09:10:58.295 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_delay": { 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_delay", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T09:10:58.296 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_no_wait": { 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T09:10:58.297 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix", 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "discovery_interval": { 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "discovery_interval", 2024-03-21T09:10:58.298 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "100", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "identifier": { 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "identifier", 2024-03-21T09:10:58.299 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.300 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.301 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.302 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.303 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.304 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.305 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.306 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_host": { 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_host", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.307 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_port": { 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_port", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10051", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.308 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_sender": { 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_sender", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.309 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: "gid": 24110, 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "y", 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: "mgr_features": 4540701547738038271, 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: "available_modules": [ 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.310 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "alerts", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.311 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.312 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.313 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.314 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.315 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.316 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_destination": { 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_destination", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.317 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email address to send alerts to", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_from_name": { 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_from_name", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "Ceph", 2024-03-21T09:10:58.318 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email From: name", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_host": { 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_host", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.319 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP server", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_password": { 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_password", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.320 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Password to authenticate with", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_port": { 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_port", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "465", 2024-03-21T09:10:58.321 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP port", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_sender": { 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_sender", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.322 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP envelope sender", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_ssl": { 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_ssl", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.323 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_user": { 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_user", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.324 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "User to authenticate as", 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "balancer", 2024-03-21T09:10:58.325 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "active": { 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "active", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T09:10:58.326 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_time": { 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_time", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0000", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.327 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_weekday": { 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_weekday", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.328 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_max_iterations": { 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_max_iterations", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "25", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "250", 2024-03-21T09:10:58.329 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_metrics": { 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_metrics", 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.330 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_step": { 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_step", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.331 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.5", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0.001", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "0.999", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "aggressiveness of optimization", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "end_time": { 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_time", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.332 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2359", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "end_weekday": { 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_weekday", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.333 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.334 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.335 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.336 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.337 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.338 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "min_score": { 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_score", 2024-03-21T09:10:58.339 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.340 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "upmap", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "crush-compat", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "none", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "read", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap", 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap-read" 2024-03-21T09:10:58.341 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Balancer mode", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_ids": { 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_ids", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.342 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.343 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_deviation": { 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_deviation", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.344 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_optimizations": { 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_optimizations", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.345 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm", 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.346 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_down_multiplier": { 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_down_multiplier", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.347 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_refresh_rate": { 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_refresh_rate", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "20", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.348 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_starting_port": { 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_starting_port", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4721", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.349 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_ptrace": { 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_ptrace", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T09:10:58.350 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_interval": { 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_interval", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T09:10:58.351 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_memory_target_ratio": { 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.7", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.352 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm_log_destination": { 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm_log_destination", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.353 INFO:teuthology.orchestra.run.smithi087.stdout: "file", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "file,syslog", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "syslog" 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "cgroups_split": { 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cgroups_split", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.354 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "config_checks_enabled": { 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_checks_enabled", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.355 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "config_dashboard": { 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_dashboard", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.356 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_alertmanager": { 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_alertmanager", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.357 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_base": { 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_base", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.358 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Container image name, without the tag", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_elasticsearch": { 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_elasticsearch", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.359 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "elasticsearch container image", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_grafana": { 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_grafana", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.360 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_haproxy": { 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_haproxy", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.361 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "HAproxy container image", 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_agent": { 2024-03-21T09:10:58.362 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_agent", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger agent container image", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_collector": { 2024-03-21T09:10:58.363 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_collector", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger collector container image", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_query": { 2024-03-21T09:10:58.364 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_query", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger query container image", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.365 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_keepalived": { 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_keepalived", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Keepalived container image", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.366 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_loki": { 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_loki", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Loki container image", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.367 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_node_exporter": { 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_node_exporter", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.368 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_nvmeof": { 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_nvmeof", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Nvme-of container image", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.369 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_prometheus": { 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_prometheus", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.370 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_promtail": { 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_promtail", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Promtail container image", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.371 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_snmp_gateway": { 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_snmp_gateway", 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.372 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SNMP Gateway container image", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "container_init": { 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_init", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.373 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "daemon_cache_timeout": { 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "daemon_cache_timeout", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.374 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "default_cephadm_command_timeout": { 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "900", 2024-03-21T09:10:58.375 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "default_registry": { 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_registry", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io", 2024-03-21T09:10:58.376 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "device_cache_timeout": { 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_cache_timeout", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1800", 2024-03-21T09:10:58.377 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache device inventory", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "device_enhanced_scan": { 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_enhanced_scan", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.378 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "facts_cache_timeout": { 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "facts_cache_timeout", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.379 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache host facts data", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "grafana_dashboards_path": { 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "grafana_dashboards_path", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.380 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "host_check_interval": { 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "host_check_interval", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.381 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to perform a host check", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "hw_monitoring": { 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hw_monitoring", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.382 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "inventory_list_all": { 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "inventory_list_all", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.383 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.384 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.385 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "log_refresh_metadata": { 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_refresh_metadata", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T09:10:58.386 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.387 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.388 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.389 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T09:10:58.390 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T09:10:58.391 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "*", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "max_count_per_host": { 2024-03-21T09:10:58.392 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_count_per_host", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of daemons per service per host", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "max_osd_draining_count": { 2024-03-21T09:10:58.393 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_osd_draining_count", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.394 INFO:teuthology.orchestra.run.smithi087.stdout: "migration_current": { 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "migration_current", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "internal - do not modify", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.395 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "root", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm-package", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "root" 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.396 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "oob_default_addr": { 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "oob_default_addr", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "169.254.1.1", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.397 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "prometheus_alerts_path": { 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus_alerts_path", 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.398 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_insecure": { 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_insecure", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.399 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_password": { 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_password", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.400 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_url": { 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_url", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.401 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_username": { 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_username", 2024-03-21T09:10:58.402 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "secure_monitoring_stack": { 2024-03-21T09:10:58.403 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "secure_monitoring_stack", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.404 INFO:teuthology.orchestra.run.smithi087.stdout: "service_discovery_port": { 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "service_discovery_port", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8765", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "cephadm service discovery port", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.405 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "ssh_config_file": { 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssh_config_file", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.406 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "use_agent": { 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_agent", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T09:10:58.407 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "use_repo_digest": { 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_repo_digest", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T09:10:58.408 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_failed_host_check": { 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_failed_host_check", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.409 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_daemons": { 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_daemons", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.410 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_hosts": { 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_hosts", 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.411 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crash", 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.412 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.413 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.414 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.415 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.416 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.417 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "retain_interval": { 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retain_interval", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "31536000", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.418 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_recent_interval": { 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_recent_interval", 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.419 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1209600", 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dashboard", 2024-03-21T09:10:58.420 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.421 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.422 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.423 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_ENABLED": { 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.424 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.425 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.426 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.427 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.428 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.429 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.430 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.431 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T09:10:58.432 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.433 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.434 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.435 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.436 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.437 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_URL": { 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_URL", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.438 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.439 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.440 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.441 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.442 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.443 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.444 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "{}", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.445 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T09:10:58.446 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.447 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T09:10:58.448 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T09:10:58.449 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.450 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.451 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.452 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.453 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.454 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.455 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.456 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.457 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.458 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8", 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T09:10:58.459 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "45", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.460 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.461 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.462 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.463 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.464 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.465 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.466 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.467 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.468 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "cross_origin_url": { 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cross_origin_url", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.469 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "crt_file": { 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crt_file", 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.470 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "debug": { 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "debug", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.471 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable/disable debug options", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "jwt_token_ttl": { 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "jwt_token_ttl", 2024-03-21T09:10:58.472 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "28800", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.473 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.474 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.475 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.476 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.477 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.478 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.479 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "motd": { 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "motd", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.480 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "The message of the day", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect_resolve_ip_addr": { 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.481 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.482 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.483 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8080", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.484 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl_server_port": { 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl_server_port", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.485 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8443", 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-03-21T09:10:58.486 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "redirect", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect" 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.487 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.488 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "url_prefix": { 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url_prefix", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.489 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.490 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_monitoring": { 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_monitoring", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "monitor device health metrics", 2024-03-21T09:10:58.491 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.492 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.493 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.494 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.495 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.496 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "mark_out_threshold": { 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mark_out_threshold", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.497 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2419200", 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_name": { 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_name", 2024-03-21T09:10:58.498 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "device_health_metrics", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.499 INFO:teuthology.orchestra.run.smithi087.stdout: "retention_period": { 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retention_period", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15552000", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain device health metrics", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.500 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_frequency": { 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_frequency", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.501 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "self_heal": { 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "self_heal", 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.502 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.503 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_threshold": { 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_threshold", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7257600", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.504 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "diskprediction_local", 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.505 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.506 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.507 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.508 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.509 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.510 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "predict_interval": { 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predict_interval", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.511 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "predictor_model": { 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predictor_model", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "prophetstor", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.512 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.513 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "influx", 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": false, 2024-03-21T09:10:58.514 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "influxdb python module not found", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "batch_size": { 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "batch_size", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5000", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T09:10:58.515 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "database": { 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "database", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "ceph", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.516 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": { 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hostname", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.517 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server hostname", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "30", 2024-03-21T09:10:58.518 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "5", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.519 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.520 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.521 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.522 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.523 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.524 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "password": { 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "password", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.525 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "password of InfluxDB server user", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "port": { 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "port", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8086", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.526 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server port", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "false", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.527 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "threads": { 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threads", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.528 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "32", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "username": { 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "username", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.529 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "username of InfluxDB server user", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "verify_ssl": { 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "verify_ssl", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "true", 2024-03-21T09:10:58.530 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "insights", 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.531 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.532 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.533 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.534 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.535 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.536 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "iostat", 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.537 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.538 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.539 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.540 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.541 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.542 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "k8sevents", 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_event_retention_days": { 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ceph_event_retention_days", 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7", 2024-03-21T09:10:58.543 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "config_check_secs": { 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_check_secs", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.544 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "10", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.545 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.546 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.547 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.548 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.549 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.550 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "localpool", 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "failure_domain": { 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "failure_domain", 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.551 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "host", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "failure domain for any created local pool", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.552 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.553 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.554 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.555 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.556 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "min_size": { 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_size", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.557 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default min_size for any created local pool", 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.558 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "num_rep": { 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "num_rep", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default replica count for any created local pool", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.559 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_num": { 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_num", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "128", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.560 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "prefix": { 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prefix", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name prefix for any created local pool", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.561 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "subtree": { 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "subtree", 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "rack", 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.562 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mds_autoscaler", 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.563 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.564 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.565 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.566 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.567 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.568 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirroring", 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.569 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.570 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.571 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.572 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.573 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.574 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "nfs", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.575 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.576 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.577 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.578 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.579 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.580 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "fail_fs": { 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "fail_fs", 2024-03-21T09:10:58.581 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.582 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.583 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.584 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.585 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.586 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.587 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator": { 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.588 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "rook", 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "test_orchestrator" 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Orchestrator backend", 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_perf_query", 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.589 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.590 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.591 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.592 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.593 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.594 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_support", 2024-03-21T09:10:58.595 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.596 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.597 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.598 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.599 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.600 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_autoscaler", 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.601 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.602 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.603 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.604 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.605 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.606 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.607 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "threshold": { 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threshold", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1.0", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.608 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "scaling threshold", 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "progress", 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_pg_recovery_event": { 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_pg_recovery_event", 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.609 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-03-21T09:10:58.610 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.611 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.612 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.613 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.614 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.615 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.616 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "max_completed_events": { 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_completed_events", 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.617 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "50", 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "number of past completed events to remember", 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.618 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long the module is going to sleep", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.619 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus", 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "cache": { 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cache", 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.620 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "exclude_perf_counters": { 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "exclude_perf_counters", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.621 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.622 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.623 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.624 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.625 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.626 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.627 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools": { 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.628 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "300", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.629 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_interval": { 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_interval", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15.0", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.630 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T09:10:58.631 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "9283", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.632 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "stale_cache_strategy": { 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stale_cache_strategy", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "log", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.633 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "default", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.634 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "default", 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "error" 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.635 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.636 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.637 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.638 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.639 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.640 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.641 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_snap_create": { 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_snap_create", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.642 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "mirror_snapshot_schedule": { 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.643 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "trash_purge_schedule": { 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "trash_purge_schedule", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.644 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "restful", 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_auth": { 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_auth", 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.645 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.646 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.647 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.648 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.649 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.650 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.651 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.652 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.653 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rgw", 2024-03-21T09:10:58.654 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.655 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.656 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.657 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.658 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.659 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rook", 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.660 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.661 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.662 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.663 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.664 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "storage_class": { 2024-03-21T09:10:58.665 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "storage_class", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "local", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.666 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "selftest", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.667 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.668 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.669 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.670 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.671 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.672 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "roption1": { 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption1", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.673 INFO:teuthology.orchestra.run.smithi087.stdout: "roption2": { 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption2", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "xyz", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.674 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption1": { 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption1", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.675 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption2": { 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption2", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.676 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption3": { 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption3", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.677 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption4": { 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption4", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.678 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption5": { 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption5", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.679 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption6": { 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption6", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.680 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption7": { 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption7", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "42", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.681 INFO:teuthology.orchestra.run.smithi087.stdout: "testkey": { 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testkey", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.682 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "testlkey": { 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testlkey", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.683 INFO:teuthology.orchestra.run.smithi087.stdout: "testnewline": { 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testnewline", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.684 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_m_granularity": { 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_m_granularity", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.685 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "dump_on_update": { 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dump_on_update", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.686 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "dump database to debug log on update", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.687 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.688 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.689 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.690 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.691 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stats", 2024-03-21T09:10:58.692 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.693 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.694 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.695 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.696 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.697 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "status", 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.698 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.699 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.700 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.701 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.702 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.703 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telegraf", 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "address": { 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "address", 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T09:10:58.704 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.705 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.706 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.707 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.708 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.709 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.710 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.711 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telemetry", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_basic": { 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_basic", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.712 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_crash": { 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_crash", 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.713 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_device": { 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_device", 2024-03-21T09:10:58.714 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_ident": { 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_ident", 2024-03-21T09:10:58.715 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_perf": { 2024-03-21T09:10:58.716 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_perf", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "contact": { 2024-03-21T09:10:58.717 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "contact", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.718 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "description": { 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "description", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.719 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "device_url": { 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_url", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.720 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.721 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "24", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "8", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.722 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "last_opt_revision": { 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "last_opt_revision", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.723 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard": { 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.724 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard_description": { 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard_description", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.725 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.726 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.727 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.728 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.729 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.730 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "organization": { 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "organization", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.731 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "proxy": { 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "proxy", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.732 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "url": { 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.733 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "test_orchestrator", 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.734 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.735 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.736 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.737 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.738 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.739 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.740 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.741 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.742 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.743 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.744 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_clones": { 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_clones", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.745 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "periodic_async_work": { 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "periodic_async_work", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.746 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Periodically check for async work", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_delay": { 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_delay", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.747 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_no_wait": { 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.748 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix", 2024-03-21T09:10:58.749 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "discovery_interval": { 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "discovery_interval", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "100", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.750 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "identifier": { 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "identifier", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.751 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.752 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.753 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.754 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.755 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.756 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.757 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_host": { 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_host", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.758 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_port": { 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_port", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10051", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.759 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_sender": { 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_sender", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.760 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "modules": [ 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "dashboard", 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "iostat", 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "nfs", 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "restful" 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.761 INFO:teuthology.orchestra.run.smithi087.stdout: "available_modules": [ 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "alerts", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.762 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How frequently to reexamine health status", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.763 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.764 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.765 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.766 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.767 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_destination": { 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_destination", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.768 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email address to send alerts to", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_from_name": { 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_from_name", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.769 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "Ceph", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Email From: name", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_host": { 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_host", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.770 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP server", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_password": { 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_password", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.771 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Password to authenticate with", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_port": { 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_port", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.772 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "465", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP port", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_sender": { 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_sender", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.773 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SMTP envelope sender", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_ssl": { 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_ssl", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.774 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use SSL to connect to SMTP server", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "smtp_user": { 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "smtp_user", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.775 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "User to authenticate as", 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "balancer", 2024-03-21T09:10:58.776 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "active": { 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "active", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically balance PGs across cluster", 2024-03-21T09:10:58.777 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_time": { 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_time", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0000", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.778 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "beginning time of day to automatically balance", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "begin_weekday": { 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "begin_weekday", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-03-21T09:10:58.779 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:10:58.780 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_max_iterations": { 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_max_iterations", 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.781 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "25", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "250", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_metrics": { 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_metrics", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.782 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "pgs,objects,bytes", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "crush_compat_step": { 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crush_compat_step", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.783 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.5", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0.001", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "0.999", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "aggressiveness of optimization", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "end_time": { 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_time", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.784 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2359", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ending time of day to automatically balance", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "end_weekday": { 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "end_weekday", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.785 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "0", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "6", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.786 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.787 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.788 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.789 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.790 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "min_score": { 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_score", 2024-03-21T09:10:58.791 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.792 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "upmap", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "crush-compat", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "none", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "read", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap-read" 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Balancer mode", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.793 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_ids": { 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_ids", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.794 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.795 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_deviation": { 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_deviation", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "deviation below which no optimization is attempted", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-03-21T09:10:58.796 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "upmap_max_optimizations": { 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "upmap_max_optimizations", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.797 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm", 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_down_multiplier": { 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_down_multiplier", 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.798 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_refresh_rate": { 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_refresh_rate", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.799 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "20", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "agent_starting_port": { 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "agent_starting_port", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.800 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4721", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_ptrace": { 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_ptrace", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.801 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_interval": { 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_interval", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.802 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to autotune daemon memory", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "autotune_memory_target_ratio": { 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "autotune_memory_target_ratio", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0.7", 2024-03-21T09:10:58.803 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm_log_destination": { 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cephadm_log_destination", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.804 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "file", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "file,syslog", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "syslog" 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "cgroups_split": { 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cgroups_split", 2024-03-21T09:10:58.805 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "config_checks_enabled": { 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_checks_enabled", 2024-03-21T09:10:58.806 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "config_dashboard": { 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_dashboard", 2024-03-21T09:10:58.807 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_alertmanager": { 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_alertmanager", 2024-03-21T09:10:58.808 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_base": { 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_base", 2024-03-21T09:10:58.809 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/ceph", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Container image name, without the tag", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_elasticsearch": { 2024-03-21T09:10:58.810 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_elasticsearch", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "elasticsearch container image", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_grafana": { 2024-03-21T09:10:58.811 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_grafana", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_haproxy": { 2024-03-21T09:10:58.812 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_haproxy", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "HAproxy container image", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_agent": { 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_agent", 2024-03-21T09:10:58.813 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger agent container image", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_collector": { 2024-03-21T09:10:58.814 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_collector", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger collector container image", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_jaeger_query": { 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_jaeger_query", 2024-03-21T09:10:58.815 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Jaeger query container image", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_keepalived": { 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_keepalived", 2024-03-21T09:10:58.816 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Keepalived container image", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_loki": { 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_loki", 2024-03-21T09:10:58.817 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Loki container image", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_node_exporter": { 2024-03-21T09:10:58.818 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_node_exporter", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_nvmeof": { 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_nvmeof", 2024-03-21T09:10:58.819 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/ceph/nvmeof:1.0.0", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Nvme-of container image", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_prometheus": { 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_prometheus", 2024-03-21T09:10:58.820 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Prometheus container image", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_promtail": { 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_promtail", 2024-03-21T09:10:58.821 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Promtail container image", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "container_image_snmp_gateway": { 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_image_snmp_gateway", 2024-03-21T09:10:58.822 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "SNMP Gateway container image", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "container_init": { 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "container_init", 2024-03-21T09:10:58.823 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Run podman/docker with `--init`", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "daemon_cache_timeout": { 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "daemon_cache_timeout", 2024-03-21T09:10:58.824 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "default_cephadm_command_timeout": { 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_cephadm_command_timeout", 2024-03-21T09:10:58.825 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "900", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "default_registry": { 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "default_registry", 2024-03-21T09:10:58.826 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "docker.io", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "device_cache_timeout": { 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_cache_timeout", 2024-03-21T09:10:58.827 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1800", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache device inventory", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "device_enhanced_scan": { 2024-03-21T09:10:58.828 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_enhanced_scan", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use libstoragemgmt during device scans", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "facts_cache_timeout": { 2024-03-21T09:10:58.829 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "facts_cache_timeout", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "seconds to cache host facts data", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "grafana_dashboards_path": { 2024-03-21T09:10:58.830 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "grafana_dashboards_path", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "host_check_interval": { 2024-03-21T09:10:58.831 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "host_check_interval", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to perform a host check", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "hw_monitoring": { 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hw_monitoring", 2024-03-21T09:10:58.832 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "inventory_list_all": { 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "inventory_list_all", 2024-03-21T09:10:58.833 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.834 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.835 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "log_refresh_metadata": { 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_refresh_metadata", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.836 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.837 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.838 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.839 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.840 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf": { 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-03-21T09:10:58.841 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "*", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.842 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "max_count_per_host": { 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_count_per_host", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.843 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of daemons per service per host", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "max_osd_draining_count": { 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_osd_draining_count", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.844 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "migration_current": { 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "migration_current", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.845 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "internal - do not modify", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "mode": { 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mode", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "root", 2024-03-21T09:10:58.846 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm-package", 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "root" 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "mode for remote execution of cephadm", 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "oob_default_addr": { 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "oob_default_addr", 2024-03-21T09:10:58.847 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "169.254.1.1", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Default address for RedFish API (oob management).", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "prometheus_alerts_path": { 2024-03-21T09:10:58.848 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus_alerts_path", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_insecure": { 2024-03-21T09:10:58.849 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_insecure", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_password": { 2024-03-21T09:10:58.850 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_password", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_url": { 2024-03-21T09:10:58.851 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_url", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.852 INFO:teuthology.orchestra.run.smithi087.stdout: "registry_username": { 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "registry_username", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.853 INFO:teuthology.orchestra.run.smithi087.stdout: "secure_monitoring_stack": { 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "secure_monitoring_stack", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.854 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "service_discovery_port": { 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "service_discovery_port", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8765", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "cephadm service discovery port", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.855 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "ssh_config_file": { 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssh_config_file", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.856 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "use_agent": { 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_agent", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.857 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "use_repo_digest": { 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "use_repo_digest", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.858 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_failed_host_check": { 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_failed_host_check", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if the host check fails", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.859 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_daemons": { 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_daemons", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.860 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_on_stray_hosts": { 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_on_stray_hosts", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.861 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crash", 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.862 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.863 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.864 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.865 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.866 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "retain_interval": { 2024-03-21T09:10:58.867 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retain_interval", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "31536000", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain crashes before pruning them", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_recent_interval": { 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_recent_interval", 2024-03-21T09:10:58.868 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1209600", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "time interval in which to warn about recent crashes", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.869 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dashboard", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.870 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_HOST": { 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_HOST", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.871 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.872 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_ENABLED": { 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_ENABLED", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.873 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.874 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "ENABLE_BROWSABLE_API": { 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ENABLE_BROWSABLE_API", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.875 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.876 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.877 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.878 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.879 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_NFS": { 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.880 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RBD": { 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.881 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "FEATURE_TOGGLE_RGW": { 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.882 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.883 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_PASSWORD": { 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_PASSWORD", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.884 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.885 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_URL": { 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_URL", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.886 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_API_USERNAME": { 2024-03-21T09:10:58.887 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_API_USERNAME", 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.888 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.889 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.889 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.889 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.889 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.889 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.889 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.890 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.891 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.891 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.891 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.891 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.891 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.891 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.892 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.893 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.894 INFO:teuthology.orchestra.run.smithi087.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-03-21T09:10:58.894 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-03-21T09:10:58.894 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.894 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.894 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.894 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "ISSUE_TRACKER_API_KEY": { 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.895 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "MULTICLUSTER_CONFIG": { 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "MULTICLUSTER_CONFIG", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.896 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "{}", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_HOST": { 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_HOST", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.897 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-03-21T09:10:58.898 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-03-21T09:10:58.899 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-03-21T09:10:58.900 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-03-21T09:10:58.901 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-03-21T09:10:58.902 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-03-21T09:10:58.903 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-03-21T09:10:58.904 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-03-21T09:10:58.905 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.906 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_ENABLED": { 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_ENABLED", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-03-21T09:10:58.907 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-03-21T09:10:58.908 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.909 INFO:teuthology.orchestra.run.smithi087.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.910 INFO:teuthology.orchestra.run.smithi087.stdout: "REST_REQUESTS_TIMEOUT": { 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "45", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.911 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ACCESS_KEY": { 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ACCESS_KEY", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.912 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "admin", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.913 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SECRET_KEY": { 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SECRET_KEY", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.914 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "RGW_API_SSL_VERIFY": { 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "RGW_API_SSL_VERIFY", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.915 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "UNSAFE_TLS_v1_2": { 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "UNSAFE_TLS_v1_2", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.916 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.917 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.918 INFO:teuthology.orchestra.run.smithi087.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.919 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "cross_origin_url": { 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cross_origin_url", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.920 INFO:teuthology.orchestra.run.smithi087.stdout: "crt_file": { 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "crt_file", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.921 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "debug": { 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "debug", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Enable/disable debug options", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.922 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "jwt_token_ttl": { 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "jwt_token_ttl", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "28800", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.923 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.924 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.925 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.926 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.927 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.928 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.929 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "motd": { 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "motd", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "The message of the day", 2024-03-21T09:10:58.930 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect_resolve_ip_addr": { 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "redirect_resolve_ip_addr", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.931 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.932 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8080", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.933 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.934 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl_server_port": { 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl_server_port", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8443", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.935 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "redirect", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "redirect" 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.936 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.937 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "url_prefix": { 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url_prefix", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.938 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_monitoring": { 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_monitoring", 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.939 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "monitor device health metrics", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.940 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.941 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.942 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.943 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.944 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "mark_out_threshold": { 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mark_out_threshold", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.945 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "2419200", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "pool_name": { 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pool_name", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.946 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "device_health_metrics", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name of pool in which to store device health metrics", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "retention_period": { 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "retention_period", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.947 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15552000", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long to retain device health metrics", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_frequency": { 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_frequency", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.948 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to scrape device health metrics", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "self_heal": { 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "self_heal", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.949 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.950 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how frequently to wake up and check device health", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "warn_threshold": { 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "warn_threshold", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.951 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7257600", 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "diskprediction_local", 2024-03-21T09:10:58.952 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.953 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.954 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.955 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.956 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.957 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "predict_interval": { 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predict_interval", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.958 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "86400", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "predictor_model": { 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "predictor_model", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.959 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "prophetstor", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.960 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "600", 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "influx", 2024-03-21T09:10:58.961 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": false, 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "influxdb python module not found", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "batch_size": { 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "batch_size", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5000", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-03-21T09:10:58.962 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "database": { 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "database", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "ceph", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.963 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "hostname": { 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "hostname", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.964 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server hostname", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "30", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "5", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.965 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.966 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.967 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.968 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.969 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.970 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "password": { 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "password", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.971 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "password of InfluxDB server user", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "port": { 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "port", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "8086", 2024-03-21T09:10:58.972 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "InfluxDB server port", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "ssl": { 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ssl", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "false", 2024-03-21T09:10:58.973 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "threads": { 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threads", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.974 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "32", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "username": { 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "username", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.975 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "username of InfluxDB server user", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "verify_ssl": { 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "verify_ssl", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "true", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.976 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "insights", 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.977 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.978 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.979 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.980 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.981 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.982 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "iostat", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.983 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.985 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.986 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.987 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.988 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "k8sevents", 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.989 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_event_retention_days": { 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "ceph_event_retention_days", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "7", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Days to hold ceph event information within local cache", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.990 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "config_check_secs": { 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "config_check_secs", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "10", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.991 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.992 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.993 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.994 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.995 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:58.996 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "localpool", 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "failure_domain": { 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "failure_domain", 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.997 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "host", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "failure domain for any created local pool", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:58.998 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:58.999 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.000 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.001 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.002 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "min_size": { 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "min_size", 2024-03-21T09:10:59.003 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default min_size for any created local pool", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "num_rep": { 2024-03-21T09:10:59.004 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "num_rep", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default replica count for any created local pool", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_num": { 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_num", 2024-03-21T09:10:59.005 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "128", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "default pg_num for any created local pool", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "prefix": { 2024-03-21T09:10:59.006 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prefix", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "name prefix for any created local pool", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "subtree": { 2024-03-21T09:10:59.007 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "subtree", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "rack", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "CRUSH level for which to create a local pool", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.008 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mds_autoscaler", 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.009 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.010 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.011 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.012 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.013 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirroring", 2024-03-21T09:10:59.014 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.015 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.016 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.017 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.018 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.019 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "nfs", 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.020 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.021 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.022 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.023 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.024 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.025 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "fail_fs": { 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "fail_fs", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.026 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.027 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.028 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.029 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.030 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.031 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator": { 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "orchestrator", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.032 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "cephadm", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "rook", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "test_orchestrator" 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Orchestrator backend", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.033 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_perf_query", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.034 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.035 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.036 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.037 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.038 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "osd_support", 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.039 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.040 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.041 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.042 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.043 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.044 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "pg_autoscaler", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.045 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.046 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.047 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.048 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.049 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.050 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "threshold": { 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "threshold", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "3.0", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1.0", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "scaling threshold", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-03-21T09:10:59.051 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "progress", 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_pg_recovery_event": { 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_pg_recovery_event", 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.052 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow the module to show pg recovery progress", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.053 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.054 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.055 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.056 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.057 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.058 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "max_completed_events": { 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_completed_events", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.059 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "50", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "number of past completed events to remember", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "sleep_interval": { 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "sleep_interval", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.060 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "5", 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "how long the module is going to sleep", 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "prometheus", 2024-03-21T09:10:59.061 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "cache": { 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "cache", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.062 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "exclude_perf_counters": { 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "exclude_perf_counters", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-03-21T09:10:59.063 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.064 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.065 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.066 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.067 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.068 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools": { 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.069 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_stats_pools_refresh_interval": { 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "300", 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.070 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "scrape_interval": { 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "scrape_interval", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15.0", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.071 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "::", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.072 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "9283", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-03-21T09:10:59.073 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "stale_cache_strategy": { 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stale_cache_strategy", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "log", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.074 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_behaviour": { 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_behaviour", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "default", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.075 INFO:teuthology.orchestra.run.smithi087.stdout: "default", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "error" 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "standby_error_status_code": { 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "standby_error_status_code", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "500", 2024-03-21T09:10:59.076 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "400", 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "599", 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.077 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.078 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.079 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.080 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.082 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_snap_create": { 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_snap_create", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.083 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "mirror_snapshot_schedule": { 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "mirror_snapshot_schedule", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.084 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "trash_purge_schedule": { 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "trash_purge_schedule", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.085 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "restful", 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "enable_auth": { 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enable_auth", 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.086 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "key_file": { 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "key_file", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.087 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.088 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.089 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.090 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.091 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.092 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "server_addr": { 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_addr", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.093 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "server_port": { 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "server_port", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.094 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rgw", 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.095 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.096 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.097 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.098 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.099 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.100 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rook", 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.101 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.102 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.103 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.104 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.105 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.106 INFO:teuthology.orchestra.run.smithi087.stdout: "storage_class": { 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "storage_class", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "local", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "selftest", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.108 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.109 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.111 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.112 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "roption1": { 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption1", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.113 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "roption2": { 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "roption2", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.114 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "xyz", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption1": { 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption1", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.115 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption2": { 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption2", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.116 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption3": { 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption3", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "float", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.117 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption4": { 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption4", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.118 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption5": { 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption5", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.119 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption6": { 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption6", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.120 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "rwoption7": { 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rwoption7", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "1", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "42", 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.121 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "testkey": { 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testkey", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.122 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "testlkey": { 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testlkey", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.123 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "testnewline": { 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "testnewline", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.124 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snap_schedule", 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "allow_m_granularity": { 2024-03-21T09:10:59.125 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "allow_m_granularity", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "allow minute scheduled snapshots", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.126 INFO:teuthology.orchestra.run.smithi087.stdout: "dump_on_update": { 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "dump_on_update", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "dump database to debug log on update", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.127 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.128 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.129 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.130 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.131 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.132 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "stats", 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.133 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.134 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.135 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.136 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.137 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "status", 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.138 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.139 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.140 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.141 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.142 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.143 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telegraf", 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "address": { 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "address", 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-03-21T09:10:59.144 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "15", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.145 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.146 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.147 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.148 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.149 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.150 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "telemetry", 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.151 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_basic": { 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_basic", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share basic cluster information (size, version)", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.152 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_crash": { 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_crash", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.153 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_device": { 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_device", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_ident": { 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_ident", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.155 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "channel_perf": { 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "channel_perf", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Share various performance metrics of a cluster", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "contact": { 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "contact", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "description": { 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "description", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.158 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "device_url": { 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "device_url", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.159 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "enabled": { 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "enabled", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.160 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "24", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "8", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.161 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "last_opt_revision": { 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "last_opt_revision", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "1", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.162 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard": { 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.163 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "leaderboard_description": { 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "leaderboard_description", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.164 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.165 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.166 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.167 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.168 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.169 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "organization": { 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "organization", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.170 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "proxy": { 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "proxy", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.171 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "url": { 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "url", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.172 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "test_orchestrator", 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.173 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.174 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.175 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.176 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.177 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.178 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.179 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.180 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.181 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.182 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.183 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "max_concurrent_clones": { 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "max_concurrent_clones", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "4", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.184 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Number of asynchronous cloner threads", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "periodic_async_work": { 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "periodic_async_work", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.185 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Periodically check for async work", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_delay": { 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_delay", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "0", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.186 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "snapshot_clone_no_wait": { 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "snapshot_clone_no_wait", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "True", 2024-03-21T09:10:59.187 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix", 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "can_run": true, 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "error_string": "", 2024-03-21T09:10:59.188 INFO:teuthology.orchestra.run.smithi087.stdout: "module_options": { 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "discovery_interval": { 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "discovery_interval", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "uint", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "100", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "identifier": { 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "identifier", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "interval": { 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "interval", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "secs", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "60", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.191 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "log_level": { 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_level", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.192 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster": { 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.193 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_cluster_level": { 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_cluster_level", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "info", 2024-03-21T09:10:59.194 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [ 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "critical", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "debug", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "error", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "info", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "warning" 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "log_to_file": { 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "log_to_file", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "bool", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 1, 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "False", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.196 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_host": { 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_host", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_port": { 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_port", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "int", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "10051", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.198 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "zabbix_sender": { 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "zabbix_sender", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "str", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "level": "advanced", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "flags": 0, 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "min": "", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "max": "", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "enum_allowed": [], 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "desc": "", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "long_desc": "", 2024-03-21T09:10:59.199 INFO:teuthology.orchestra.run.smithi087.stdout: "tags": [], 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "see_also": [] 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "services": { 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "dashboard": "https://172.21.15.87:8443/" 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "always_on_modules": { 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "octopus": [ 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-03-21T09:10:59.200 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "pacific": [ 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-03-21T09:10:59.201 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "quincy": [ 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-03-21T09:10:59.202 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "reef": [ 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-03-21T09:10:59.203 INFO:teuthology.orchestra.run.smithi087.stdout: ], 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "squid": [ 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "balancer", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "crash", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "devicehealth", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "orchestrator", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "pg_autoscaler", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "progress", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "rbd_support", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "status", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "telemetry", 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "volumes" 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.204 INFO:teuthology.orchestra.run.smithi087.stdout: "last_failure_osd_epoch": 3, 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "active_clients": [ 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "devicehealth", 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 671247144 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "libcephsqlite", 2024-03-21T09:10:59.205 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 1056206501 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "rbd_support", 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-03-21T09:10:59.206 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 924709719 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: "name": "volumes", 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: "addrvec": [ 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: { 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: "type": "v2", 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: "addr": "172.21.15.87:0", 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: "nonce": 3098728466 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:59.207 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:10:59.208 INFO:teuthology.orchestra.run.smithi087.stdout: ] 2024-03-21T09:10:59.208 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f64e00a2f20 msgr2=0x7f64e00a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f64e00a2f20 0x7f64e00a53e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f64f4101610 tx=0x7f64ec009500 comp rx=0 tx=0).stop 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 msgr2=0x7f64f419eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f419eed0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f64e4005ab0 tx=0x7f64e402c6c0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 shutdown_connections 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f64e00a2f20 0x7f64e00a53e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f64f4102170 0x7f64f419eed0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.209 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f64f41017b0 0x7f64f4197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 --2- 172.21.15.87:0/619019988 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f64f41005c0 0x7f64f4197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:10:59.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 >> 172.21.15.87:0/619019988 conn(0x7f64f40fbb30 msgr2=0x7f64f4104ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:10:59.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 shutdown_connections 2024-03-21T09:10:59.210 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:10:57.854+0000 7f64f989a640 1 -- 172.21.15.87:0/619019988 wait complete. 2024-03-21T09:10:59.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:57 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1189082329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:57 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1177133020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.210 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:57 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1360589334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:57 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1189082329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:57 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1177133020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:57 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1360589334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:58 smithi100 ceph-mon[36306]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 89 MiB/s wr, 848 op/s 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:10:58 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/619019988' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:57 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1189082329' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:57 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1177133020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:57 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1360589334' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:58 smithi105 ceph-mon[35993]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 89 MiB/s wr, 848 op/s 2024-03-21T09:10:59.211 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:10:58 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/619019988' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T09:10:59.217 DEBUG:tasks.check_counter:Getting stats from a 2024-03-21T09:10:59.218 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.a perf dump 2024-03-21T09:10:59.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:58 smithi087 ceph-mon[32385]: pgmap v767: 97 pgs: 97 active+clean; 10 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 45 MiB/s rd, 89 MiB/s wr, 848 op/s 2024-03-21T09:10:59.218 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:10:58 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/619019988' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-03-21T09:10:59.720 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:00.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:00 smithi105 ceph-mon[35993]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 107 MiB/s wr, 1.05k op/s 2024-03-21T09:11:01.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:00 smithi100 ceph-mon[36306]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 107 MiB/s wr, 1.05k op/s 2024-03-21T09:11:01.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:00 smithi087 ceph-mon[32385]: pgmap v768: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 81 MiB/s rd, 107 MiB/s wr, 1.05k op/s 2024-03-21T09:11:02.062 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 1888, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 1890, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 565029, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 600385, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 6, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.476581279, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.072470588, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.080074916, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.305271089, 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:02.063 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1888, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.313406816, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000165999 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:02.064 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 5, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 200, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1710, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 511694, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 8, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.020857738, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.016552978, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002434359, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000066719, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 198, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.037787357, 2024-03-21T09:11:02.065 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000190845 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 510672 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 276, 2024-03-21T09:11:02.066 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 268, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 792140, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 157182, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.057792946, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.021999918, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.025490476, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.010588594, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 268, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.043198008, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000161186 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.067 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 791872, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 156864 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2024-03-21T09:11:02.068 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 3777, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.179864152, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000047620 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.069 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:02.070 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 1, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 0, 2024-03-21T09:11:02.071 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:02.072 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:02.073 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 0 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2024-03-21T09:11:02.074 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:02.075 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.076 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "evlrg": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 2, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 2, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "segmjr": 2, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2024-03-21T09:11:02.077 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 0, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 4194304, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 4194892, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 4194352, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 2 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 3, 2024-03-21T09:11:02.078 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 3, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 1, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 1, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 0, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 49980, 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 215420 2024-03-21T09:11:02.079 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.080 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2024-03-21T09:11:02.081 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.082 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.083 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.084 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.085 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2024-03-21T09:11:02.086 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.087 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.088 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.089 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.090 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.091 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 0, 2024-03-21T09:11:02.092 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:02.093 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:02.094 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-03-21T09:11:02.095 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 190231, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 53, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:02.096 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 156, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 8320, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 13, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-03-21T09:11:02.097 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 1892, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 21, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 3, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 1892, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1892, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 5.065887524, 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.002677530 2024-03-21T09:11:02.098 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 1892, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 1893 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 1892, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 1892, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 0, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 943, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 0, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 947, 2024-03-21T09:11:02.099 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 1, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:02.100 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2024-03-21T09:11:02.101 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 88, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 2, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 5, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 8, 2024-03-21T09:11:02.102 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 3, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 0, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 2, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 0 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 0, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.103 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 0 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 0, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:02.104 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 2165, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1147054, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 2165, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 2165, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1147054, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.105 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 1892, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 593, 2024-03-21T09:11:02.106 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 1892, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 593, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:02.107 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 1892, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 1892, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 1892, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1892, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:02.108 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.109 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x558795bb06e0": { 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:02.110 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 0, 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 0, 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:02.111 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T09:11:02.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:02 smithi105 ceph-mon[35993]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 83 MiB/s wr, 1.03k op/s 2024-03-21T09:11:03.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:02 smithi100 ceph-mon[36306]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 83 MiB/s wr, 1.03k op/s 2024-03-21T09:11:03.098 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:02 smithi087 ceph-mon[32385]: pgmap v769: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 83 MiB/s wr, 1.03k op/s 2024-03-21T09:11:03.129 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-03-21T09:11:03.129 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-03-21T09:11:03.129 DEBUG:tasks.check_counter:Getting stats from d 2024-03-21T09:11:03.129 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.d perf dump 2024-03-21T09:11:03.389 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:04.856 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:04 smithi087 ceph-mon[32385]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 103 MiB/s wr, 1.31k op/s 2024-03-21T09:11:04.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:04 smithi105 ceph-mon[35993]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 103 MiB/s wr, 1.31k op/s 2024-03-21T09:11:05.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:04 smithi100 ceph-mon[36306]: pgmap v770: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 80 MiB/s rd, 103 MiB/s wr, 1.31k op/s 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 268, 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 263, 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 767744, 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 158352, 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2024-03-21T09:11:05.137 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.090750376, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.024940180, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.026132256, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.039492578, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 262, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.049560787, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000189163 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.138 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 767744, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 158352 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 6, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 204, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 3344, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 191392, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.021661035, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.015458537, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002087978, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:11:05.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 203, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.041546751, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000204663 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 191392 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:05.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.000251417, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000180972, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-03-21T09:11:05.142 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:05.143 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:05.144 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148704, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 29, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-03-21T09:11:05.145 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:05.146 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 270, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 729901, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 270, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 270, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 729901, 2024-03-21T09:11:05.147 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:05.148 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:05.148 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:05.148 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:05.148 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:05.148 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:05.148 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T09:11:05.848 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-03-21T09:11:05.849 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-03-21T09:11:05.849 DEBUG:tasks.check_counter:Getting stats from g 2024-03-21T09:11:05.849 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.g perf dump 2024-03-21T09:11:06.116 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:06.539 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:06 smithi087 ceph-mon[32385]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 74 MiB/s wr, 921 op/s 2024-03-21T09:11:06.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:06 smithi105 ceph-mon[35993]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 74 MiB/s wr, 921 op/s 2024-03-21T09:11:06.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:06 smithi100 ceph-mon[36306]: pgmap v771: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 69 MiB/s rd, 74 MiB/s wr, 921 op/s 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 946, 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 948, 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 317801, 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 300829, 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 7, 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:08.139 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.182302840, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.055015867, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.042872668, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.067707941, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 946, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.294367658, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000311170 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.140 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 2057, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2049, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1246908, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 724209, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 3, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.433801312, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.066772370, 2024-03-21T09:11:08.141 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.095785857, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.259766475, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2049, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.267058407, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000130335 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.142 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 792416, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 158112 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 5, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 202, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 1668, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 515189, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 7, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.018773758, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.014228537, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002486161, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000053098, 2024-03-21T09:11:08.143 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 200, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.016398066, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000081990 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 514224 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-03-21T09:11:08.144 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 1, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-MDSRank": { 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 4572, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.187158987, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000040935 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "queue_len": 0, 2024-03-21T09:11:08.145 INFO:teuthology.orchestra.run.smithi087.stdout: "complete_latency": { 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "mds": { 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "request": 0, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "reply": 0, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "reply_latency": { 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.146 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_reply": 0, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "forward": 0, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_commit": 0, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_split": 0, 2024-03-21T09:11:08.147 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_merge": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_top": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_bottom": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_pinned": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_expired": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:08.148 INFO:teuthology.orchestra.run.smithi087.stdout: "caps": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "subtrees": 1, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_hit": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_forward": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_discover": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "traverse_lock": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "load_cent": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "q": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "exported": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "exported_inodes": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "imported": 0, 2024-03-21T09:11:08.149 INFO:teuthology.orchestra.run.smithi087.stdout: "imported_inodes": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rfiles": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rbytes": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "root_rsnaps": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:08.150 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "process_request_cap_release": 0 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.151 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_cache": { 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_try_discover": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_send_discover": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_created": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_enqueued": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "strays_migrated": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:08.152 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:08.153 INFO:teuthology.orchestra.run.smithi087.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:08.153 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_started": 0, 2024-03-21T09:11:08.153 INFO:teuthology.orchestra.run.smithi087.stdout: "recovery_completed": 0, 2024-03-21T09:11:08.153 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:08.153 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:08.153 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_flush": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "ireq_inodestats": 0 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_log": { 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "evlrg": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "evadd": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "evex": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "evtrm": 0, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "ev": 5, 2024-03-21T09:11:08.154 INFO:teuthology.orchestra.run.smithi087.stdout: "evexg": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "evexd": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "segadd": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "segex": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "segtrm": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "seg": 2, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "segmjr": 2, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "segexg": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "segexd": 0, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "expos": 4194304, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "wrpos": 4195732, 2024-03-21T09:11:08.155 INFO:teuthology.orchestra.run.smithi087.stdout: "rdpos": 4195591, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "jlat": { 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "replayed": 5 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_mem": { 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "ino": 3, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "ino+": 3, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "ino-": 0, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "dir": 1, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "dir+": 1, 2024-03-21T09:11:08.156 INFO:teuthology.orchestra.run.smithi087.stdout: "dir-": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "dn": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "dn+": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "dn-": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "cap": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "cap+": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "cap-": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "rss": 49436, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "heap": 215420 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_server": { 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_client_request": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_request": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_client_session": 0, 2024-03-21T09:11:08.157 INFO:teuthology.orchestra.run.smithi087.stdout: "handle_peer_request": 0, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "req_create_latency": { 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getattr_latency": { 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.158 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "req_link_latency": { 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookup_latency": { 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.159 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupino_latency": { 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupname_latency": { 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.160 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "req_lssnap_latency": { 2024-03-21T09:11:08.161 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mkdir_latency": { 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.162 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mknod_latency": { 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: "req_mksnap_latency": { 2024-03-21T09:11:08.163 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "req_open_latency": { 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "req_readdir_latency": { 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rename_latency": { 2024-03-21T09:11:08.164 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.165 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmdir_latency": { 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.166 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setattr_latency": { 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.167 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setlayout_latency": { 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "req_setxattr_latency": { 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "req_symlink_latency": { 2024-03-21T09:11:08.168 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: "req_unlink_latency": { 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.169 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_sessions": { 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "session_count": 1, 2024-03-21T09:11:08.170 INFO:teuthology.orchestra.run.smithi087.stdout: "session_add": 2, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "session_remove": 1, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_open": 2, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "sessions_stale": 0, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "total_load": 0, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "average_load": 0, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "avg_session_uptime": 3102, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-03-21T09:11:08.171 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:08.172 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:08.173 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:08.174 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 198187, 2024-03-21T09:11:08.175 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 55, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:08.176 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 156, 2024-03-21T09:11:08.177 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:08.177 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:08.177 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:08.177 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-03-21T09:11:08.177 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 8320, 2024-03-21T09:11:08.177 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 13, 2024-03-21T09:11:08.178 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:08.178 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-03-21T09:11:08.178 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:08.178 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-03-21T09:11:08.178 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.179 INFO:teuthology.orchestra.run.smithi087.stdout: "objecter": { 2024-03-21T09:11:08.179 INFO:teuthology.orchestra.run.smithi087.stdout: "op_active": 0, 2024-03-21T09:11:08.179 INFO:teuthology.orchestra.run.smithi087.stdout: "op_laggy": 0, 2024-03-21T09:11:08.179 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send": 2729, 2024-03-21T09:11:08.179 INFO:teuthology.orchestra.run.smithi087.stdout: "op_send_bytes": 21, 2024-03-21T09:11:08.179 INFO:teuthology.orchestra.run.smithi087.stdout: "op_resend": 3, 2024-03-21T09:11:08.180 INFO:teuthology.orchestra.run.smithi087.stdout: "op_reply": 2729, 2024-03-21T09:11:08.180 INFO:teuthology.orchestra.run.smithi087.stdout: "op_latency": { 2024-03-21T09:11:08.180 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2729, 2024-03-21T09:11:08.180 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 10.794760355, 2024-03-21T09:11:08.180 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.003955573 2024-03-21T09:11:08.180 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.181 INFO:teuthology.orchestra.run.smithi087.stdout: "op_inflight": 0, 2024-03-21T09:11:08.181 INFO:teuthology.orchestra.run.smithi087.stdout: "oplen_avg": { 2024-03-21T09:11:08.181 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 2729, 2024-03-21T09:11:08.181 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 2730 2024-03-21T09:11:08.181 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.181 INFO:teuthology.orchestra.run.smithi087.stdout: "op": 2729, 2024-03-21T09:11:08.182 INFO:teuthology.orchestra.run.smithi087.stdout: "op_r": 2729, 2024-03-21T09:11:08.182 INFO:teuthology.orchestra.run.smithi087.stdout: "op_w": 0, 2024-03-21T09:11:08.182 INFO:teuthology.orchestra.run.smithi087.stdout: "op_rmw": 0, 2024-03-21T09:11:08.182 INFO:teuthology.orchestra.run.smithi087.stdout: "op_pg": 0, 2024-03-21T09:11:08.182 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_stat": 1776, 2024-03-21T09:11:08.182 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_create": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_read": 951, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_write": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writefull": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_writesame": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_append": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_zero": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_truncate": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_delete": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_mapext": 0, 2024-03-21T09:11:08.183 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_getxattr": 1, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_setxattr": 0, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:08.184 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_call": 0, 2024-03-21T09:11:08.185 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_watch": 0, 2024-03-21T09:11:08.185 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_notify": 0, 2024-03-21T09:11:08.185 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:08.185 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls": 0, 2024-03-21T09:11:08.185 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:08.185 INFO:teuthology.orchestra.run.smithi087.stdout: "osdop_other": 0, 2024-03-21T09:11:08.186 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_active": 0, 2024-03-21T09:11:08.186 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_send": 0, 2024-03-21T09:11:08.186 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_resend": 0, 2024-03-21T09:11:08.186 INFO:teuthology.orchestra.run.smithi087.stdout: "linger_ping": 0, 2024-03-21T09:11:08.186 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_active": 0, 2024-03-21T09:11:08.186 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_send": 0, 2024-03-21T09:11:08.187 INFO:teuthology.orchestra.run.smithi087.stdout: "poolop_resend": 0, 2024-03-21T09:11:08.187 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_active": 0, 2024-03-21T09:11:08.187 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_send": 0, 2024-03-21T09:11:08.187 INFO:teuthology.orchestra.run.smithi087.stdout: "poolstat_resend": 0, 2024-03-21T09:11:08.187 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_active": 0, 2024-03-21T09:11:08.187 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_send": 0, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "statfs_resend": 0, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "command_active": 0, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "command_send": 0, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "command_resend": 0, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "map_epoch": 88, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "map_full": 0, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "map_inc": 2, 2024-03-21T09:11:08.188 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_sessions": 6, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_open": 8, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_session_close": 2, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_laggy": 0, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_wr": 0, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_rd": 2, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_del": 0 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "oft": { 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_objs": 0, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:08.189 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_updates": 0, 2024-03-21T09:11:08.190 INFO:teuthology.orchestra.run.smithi087.stdout: "omap_total_removes": 0 2024-03-21T09:11:08.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.190 INFO:teuthology.orchestra.run.smithi087.stdout: "osd-slow-ops": { 2024-03-21T09:11:08.190 INFO:teuthology.orchestra.run.smithi087.stdout: "slow_ops_count": 0 2024-03-21T09:11:08.190 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.193 INFO:teuthology.orchestra.run.smithi087.stdout: "purge_queue": { 2024-03-21T09:11:08.194 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:08.194 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:08.194 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing": 0, 2024-03-21T09:11:08.194 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:08.194 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_executed": 0, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 3004, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 1278938, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:08.195 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 3004, 2024-03-21T09:11:08.196 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:08.196 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:08.196 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 3004, 2024-03-21T09:11:08.196 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1278938, 2024-03-21T09:11:08.196 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:08.196 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:08.197 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 2729, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 1433, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 2729, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 1433, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 1024, 2024-03-21T09:11:08.198 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 2729, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 2729, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 2729, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 2729, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.199 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.200 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-write_buf_throttle-0x55cfb48206e0": { 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 3758096384, 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 0, 2024-03-21T09:11:08.201 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 0, 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:08.202 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:08.203 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:08.203 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:08.203 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.203 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:08.203 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T09:11:08.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:08 smithi087 ceph-mon[32385]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 61 MiB/s wr, 695 op/s 2024-03-21T09:11:08.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:08 smithi105 ceph-mon[35993]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 61 MiB/s wr, 695 op/s 2024-03-21T09:11:08.988 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-03-21T09:11:08.988 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-03-21T09:11:08.988 DEBUG:tasks.check_counter:Getting stats from j 2024-03-21T09:11:08.989 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.j perf dump 2024-03-21T09:11:09.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:08 smithi100 ceph-mon[36306]: pgmap v772: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 50 MiB/s rd, 61 MiB/s wr, 695 op/s 2024-03-21T09:11:09.241 INFO:teuthology.orchestra.run.smithi087.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.a/config 2024-03-21T09:11:10.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:10 smithi087 ceph-mon[32385]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 53 MiB/s wr, 597 op/s 2024-03-21T09:11:10.942 INFO:teuthology.orchestra.run.smithi087.stdout:{ 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 0, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 2, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 0, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 0, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 0, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.001133324, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.000000000, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.000273859, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:11:10.943 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:10.944 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 271, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 266, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 768656, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 159904, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 2, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 1, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.062051220, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.019607465, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.025312026, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.018409434, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 265, 2024-03-21T09:11:10.945 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.057927227, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000218593 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 768656, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 159904 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_messages": 6, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages": 206, 2024-03-21T09:11:10.946 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_bytes": 3344, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_bytes": 191744, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_total_time": 0.022781013, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_send_time": 0.015056941, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_recv_time": 0.002010879, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 205, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.028308245, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000138089 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.947 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "msgr_send_encrypted_bytes": 191744 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "cct": { 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "total_workers": 0, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "unhealthy_workers": 0 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "mempool": { 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:10.948 INFO:teuthology.orchestra.run.smithi087.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:10.949 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:10.950 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_items": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_bytes": 148721, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_anon_items": 30, 2024-03-21T09:11:10.951 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "pgmap_items": 0, 2024-03-21T09:11:10.952 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_bytes": 928, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "mds_co_items": 2, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_1_items": 0, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "unittest_2_items": 0 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: }, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "val": 0, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "max": 104857600, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "get_started": 0, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "get": 273, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "get_sum": 730394, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:10.953 INFO:teuthology.orchestra.run.smithi087.stdout: "get_or_fail_success": 273, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "take": 0, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "take_sum": 0, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "put": 273, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "put_sum": 730394, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "wait": { 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgcount": 0, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "sum": 0.000000000, 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: "avgtime": 0.000000000 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout: } 2024-03-21T09:11:10.954 INFO:teuthology.orchestra.run.smithi087.stdout:} 2024-03-21T09:11:10.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:10 smithi105 ceph-mon[35993]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 53 MiB/s wr, 597 op/s 2024-03-21T09:11:11.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:10 smithi100 ceph-mon[36306]: pgmap v773: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 53 MiB/s wr, 597 op/s 2024-03-21T09:11:11.721 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-03-21T09:11:11.721 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-03-21T09:11:11.721 DEBUG:tasks.check_counter:Getting stats from b 2024-03-21T09:11:11.721 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.b perf dump 2024-03-21T09:11:12.021 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:11:12.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:12 smithi105 ceph-mon[35993]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 34 MiB/s wr, 418 op/s 2024-03-21T09:11:12.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:12 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:11:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:12 smithi100 ceph-mon[36306]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 34 MiB/s wr, 418 op/s 2024-03-21T09:11:13.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:12 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:11:13.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:12 smithi087 ceph-mon[32385]: pgmap v774: 97 pgs: 97 active+clean; 10 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 34 MiB/s wr, 418 op/s 2024-03-21T09:11:13.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:12 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-21T09:11:13.880 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:11:13.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:11:13.881 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:13 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:11:13.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:11:13.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:11:13.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:13 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:11:14.006 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T09:11:14.006 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:14.006 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 45, 2024-03-21T09:11:14.006 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 48, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 12695, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 60634, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 14, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.014425784, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.002556306, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.009787482, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000509902, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 47, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.015958097, 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000339533 2024-03-21T09:11:14.007 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 34, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000007359, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000216 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 6368, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 5040 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1169, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1048, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 279225, 2024-03-21T09:11:14.008 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 755249, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 9, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 5, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.096857195, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.037797580, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.068060836, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.005407780, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1046, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.074260579, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000070994 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1900, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000479160, 2024-03-21T09:11:14.009 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000252 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 430272 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 325, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 311, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 827899, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 173848, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 8, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:14.010 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.043638860, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.014185229, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.023933644, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.007706398, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 310, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.021109352, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000068094 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 7, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000001073, 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000153 2024-03-21T09:11:14.011 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 825088, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 163664 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 43, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001521651, 2024-03-21T09:11:14.012 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000035387 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000009509, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000004754 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 2, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 1, 2024-03-21T09:11:14.013 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000773621, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000773621 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 1, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 14, 2024-03-21T09:11:14.014 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 14, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 14, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 1, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 1, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 2, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 3, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 1, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 1, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T09:11:14.015 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T09:11:14.016 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 2, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 3, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:14.017 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 2, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 1 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 2, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T09:11:14.018 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T09:11:14.019 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 5, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 5, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 2, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T09:11:14.020 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4195732, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.008503094, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002125773 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 17, 2024-03-21T09:11:14.021 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 17, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 15, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 16, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 1, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 14, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 14, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 1, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 1, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 50780, 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 223612 2024-03-21T09:11:14.022 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 3, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 2, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 15, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.023 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000773621, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000773621 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.024 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.025 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T09:11:14.026 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.027 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.028 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.029 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.030 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T09:11:14.031 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.032 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.033 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.034 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.035 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 1, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 1, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 845, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T09:11:14.036 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:14.037 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:14.038 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 254407, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 80, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T09:11:14.039 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 156, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 66949, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 288, 2024-03-21T09:11:14.040 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 42, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 12135, 2024-03-21T09:11:14.041 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 32, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 42, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 42, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.251993509, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005999845 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 42, 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 66 2024-03-21T09:11:14.042 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 42, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 0, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 42, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 0, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 4, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 7, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T09:11:14.043 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 6, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T09:11:14.044 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T09:11:14.045 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 88, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T09:11:14.046 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 2, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 9, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 14, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 5, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 18, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 1 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T09:11:14.047 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.048 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 1535, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 962076, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 1535, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1535, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 962076, 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:14.049 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 42, 2024-03-21T09:11:14.050 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 12135, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 42, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 12135, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:14.051 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 42, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 42, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 42, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 42, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.052 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.053 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x561f266c8500": { 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 5, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1428, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 5, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:14.054 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 4, 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1428, 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:14.055 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T09:11:14.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-21T09:11:14.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-21T09:11:14.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:13 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' 2024-03-21T09:11:14.766 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:14 smithi100 ceph-mon[36306]: pgmap v775: 97 pgs: 97 active+clean; 9.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 55 MiB/s wr, 578 op/s 2024-03-21T09:11:14.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:14 smithi105 ceph-mon[35993]: pgmap v775: 97 pgs: 97 active+clean; 9.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 55 MiB/s wr, 578 op/s 2024-03-21T09:11:15.088 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-03-21T09:11:15.088 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-03-21T09:11:15.088 DEBUG:tasks.check_counter:Getting stats from e 2024-03-21T09:11:15.089 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.e perf dump 2024-03-21T09:11:15.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:14 smithi087 ceph-mon[32385]: pgmap v775: 97 pgs: 97 active+clean; 9.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 55 MiB/s wr, 578 op/s 2024-03-21T09:11:15.358 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:11:16.871 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:16 smithi100 ceph-mon[36306]: pgmap v776: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 40 MiB/s wr, 359 op/s 2024-03-21T09:11:16.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:16 smithi105 ceph-mon[35993]: pgmap v776: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 40 MiB/s wr, 359 op/s 2024-03-21T09:11:17.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:16 smithi087 ceph-mon[32385]: pgmap v776: 97 pgs: 97 active+clean; 8.9 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.6 KiB/s rd, 40 MiB/s wr, 359 op/s 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 21, 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 23, 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 5221, 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 52834, 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 14, 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-21T09:11:17.500 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.004722612, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.000233273, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.002920340, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000338034, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 21, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.011853356, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000564445 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:17.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4, 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000447, 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000111 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 18, 2024-03-21T09:11:17.502 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 218, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 6635, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 526023, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 10, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.013375578, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.008167058, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.003162498, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000341374, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 217, 2024-03-21T09:11:17.503 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.021498042, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000099069 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 520352 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 730, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 885, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 902078, 2024-03-21T09:11:17.504 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 269915, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.091632718, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.032910523, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.047866427, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.012570294, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 884, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.048464468, 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000054824 2024-03-21T09:11:17.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 975, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000314896, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000322 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 814000, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 166512 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.506 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 43, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.001230226, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000028609 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000005964, 2024-03-21T09:11:17.507 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000002982 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-03-21T09:11:17.508 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 11, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 10, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 10, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 10, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 1, 2024-03-21T09:11:17.509 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T09:11:17.510 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 4, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 234, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:17.511 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:17.512 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:17.512 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:17.512 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:17.512 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:17.512 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:17.512 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.513 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 1, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T09:11:17.514 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T09:11:17.515 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 2, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 2, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 2, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4194892, 2024-03-21T09:11:17.516 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 0, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.002309148, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.002309148 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 0 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 13, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 13, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 11, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 12, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 1, 2024-03-21T09:11:17.517 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 10, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 10, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 51980, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 223612 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-03-21T09:11:17.518 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.520 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.521 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.522 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.524 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.525 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.527 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.528 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T09:11:17.529 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:17.530 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-03-21T09:11:17.531 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:17.532 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:17.533 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:17.534 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 229943, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 70, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T09:11:17.535 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 156, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 50040, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 198, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:17.536 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 35, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 10710, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 33, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 35, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 35, 2024-03-21T09:11:17.537 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.197994883, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.005656996 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 35, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 57 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 35, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 0, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 35, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T09:11:17.538 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 10, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 1, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 5, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 6, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 0, 2024-03-21T09:11:17.539 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 20, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T09:11:17.540 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T09:11:17.541 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 88, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 2, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 8, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 13, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 5, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 14, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 0, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 1 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 1, 2024-03-21T09:11:17.542 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T09:11:17.543 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 765, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 826094, 2024-03-21T09:11:17.544 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 765, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 765, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 826094, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.545 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 35, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 10710, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 35, 2024-03-21T09:11:17.546 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 10710, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:17.547 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 35, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 35, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 35, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 35, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.548 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.549 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x565050f54500": { 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 2, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 588, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 2, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:17.550 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 1, 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 588, 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:17.551 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T09:11:18.424 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-03-21T09:11:18.424 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-03-21T09:11:18.424 DEBUG:tasks.check_counter:Getting stats from h 2024-03-21T09:11:18.425 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.h perf dump 2024-03-21T09:11:18.681 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:11:18.956 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:18 smithi100 ceph-mon[36306]: pgmap v777: 97 pgs: 97 active+clean; 8.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 MiB/s wr, 307 op/s 2024-03-21T09:11:18.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:18 smithi105 ceph-mon[35993]: pgmap v777: 97 pgs: 97 active+clean; 8.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 MiB/s wr, 307 op/s 2024-03-21T09:11:19.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:18 smithi087 ceph-mon[32385]: pgmap v777: 97 pgs: 97 active+clean; 8.5 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 35 MiB/s wr, 307 op/s 2024-03-21T09:11:20.474 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T09:11:20.474 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:20.474 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 2789, 2024-03-21T09:11:20.474 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 2791, 2024-03-21T09:11:20.474 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 790541, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 886909, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.219503550, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.060418512, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.056131953, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.088537146, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2789, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.091781380, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000032908 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.475 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 8, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 207, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 3900, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 520874, 2024-03-21T09:11:20.476 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.012046087, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.008412162, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.001832687, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.000072898, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 206, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.014737510, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000071541 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.477 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 520176 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 286, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 273, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 819084, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 160782, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 2, 2024-03-21T09:11:20.478 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.040781240, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.013540292, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.021158552, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.007655489, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 272, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.010869264, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000039960 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.479 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 818816, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 160464 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4619, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.128849092, 2024-03-21T09:11:20.480 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000027895 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-03-21T09:11:20.481 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 0, 2024-03-21T09:11:20.482 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 1, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:20.483 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:20.484 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T09:11:20.485 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T09:11:20.486 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T09:11:20.487 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 5, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T09:11:20.488 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4195732, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 4195591, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 5 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 3, 2024-03-21T09:11:20.489 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 3, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 1, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 1, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 48460, 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 215420 2024-03-21T09:11:20.490 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T09:11:20.491 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T09:11:20.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.493 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.495 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.498 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.499 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.500 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.501 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.502 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.503 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.504 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.505 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 1, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 1, 2024-03-21T09:11:20.506 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 2, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 3532, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:20.507 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:20.508 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:20.509 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 181367, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 47, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:20.510 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 156, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 8320, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 13, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.511 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 2792, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 21, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 3, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 2792, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2792, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 8.802779860, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.003152858 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T09:11:20.512 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2792, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 2793 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 2792, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 2792, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 1817, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 973, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-03-21T09:11:20.513 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 1, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:20.514 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T09:11:20.515 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 88, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 2, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 4, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 5, 2024-03-21T09:11:20.516 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 1, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T09:11:20.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:20.518 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 3079, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1318772, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 3079, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 3079, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1318772, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.519 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 2792, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 1433, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2792, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1433, 2024-03-21T09:11:20.520 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 2792, 2024-03-21T09:11:20.521 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 2792, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2792, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2792, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:20.522 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:20.523 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x56063dbca6e0": { 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:20.524 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:20.525 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.525 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:20.525 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T09:11:20.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:20 smithi100 ceph-mon[36306]: pgmap v778: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 35 MiB/s wr, 315 op/s 2024-03-21T09:11:20.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:20 smithi105 ceph-mon[35993]: pgmap v778: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 35 MiB/s wr, 315 op/s 2024-03-21T09:11:21.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:20 smithi087 ceph-mon[32385]: pgmap v778: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 35 MiB/s wr, 315 op/s 2024-03-21T09:11:21.512 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-03-21T09:11:21.512 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-03-21T09:11:21.512 DEBUG:tasks.check_counter:Getting stats from k 2024-03-21T09:11:21.512 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.k perf dump 2024-03-21T09:11:21.767 INFO:teuthology.orchestra.run.smithi100.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.b/config 2024-03-21T09:11:22.933 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:22 smithi100 ceph-mon[36306]: pgmap v779: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 35 MiB/s wr, 315 op/s 2024-03-21T09:11:22.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:22 smithi105 ceph-mon[35993]: pgmap v779: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 35 MiB/s wr, 315 op/s 2024-03-21T09:11:23.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:22 smithi087 ceph-mon[32385]: pgmap v779: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 35 MiB/s wr, 315 op/s 2024-03-21T09:11:23.491 INFO:teuthology.orchestra.run.smithi100.stdout:{ 2024-03-21T09:11:23.491 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:23.491 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 288, 2024-03-21T09:11:23.491 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 280, 2024-03-21T09:11:23.491 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 796066, 2024-03-21T09:11:23.491 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 164356, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 6, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 1, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.042933850, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.014004748, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.021919414, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.008148155, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 280, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.012853031, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000045903 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.492 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 795232, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 164032 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 873, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1078, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 214694, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 710581, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 5, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.101842878, 2024-03-21T09:11:23.493 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.026935379, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.024238668, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.042559945, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1076, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.042602908, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000039593 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.494 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 160, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 433280 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_messages": 1898, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages": 1900, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_bytes": 568935, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_bytes": 603622, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_created_connections": 4, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_active_connections": 3, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_total_time": 0.178703785, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_send_time": 0.054527305, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_recv_time": 0.047706667, 2024-03-21T09:11:23.495 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_running_fast_dispatch_time": 0.062692942, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 1898, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.083911733, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000044210 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:23.496 INFO:teuthology.orchestra.run.smithi100.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "cct": { 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "total_workers": 1, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "unhealthy_workers": 0 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-MDSRank": { 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 4647, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.138116031, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000029721 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.497 INFO:teuthology.orchestra.run.smithi100.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "queue_len": 0, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "complete_latency": { 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "mds": { 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "request": 0, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "reply": 0, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "reply_latency": { 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.498 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_reply": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "forward": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_commit": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_split": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_merge": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_top": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_bottom": 0, 2024-03-21T09:11:23.499 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_pinned": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_expired": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "caps": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "subtrees": 1, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_hit": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_forward": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_discover": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "traverse_lock": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "load_cent": 0, 2024-03-21T09:11:23.500 INFO:teuthology.orchestra.run.smithi100.stdout: "q": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "exported": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "exported_inodes": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "imported": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "imported_inodes": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rfiles": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rbytes": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "root_rsnaps": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:23.501 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:23.502 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "process_request_cap_release": 0 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_cache": { 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_try_discover": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_send_discover": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_created": 0, 2024-03-21T09:11:23.503 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_enqueued": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "strays_migrated": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_started": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "recovery_completed": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_flush": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "ireq_inodestats": 0 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.504 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_log": { 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "evlrg": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "evadd": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "evex": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "evtrm": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "ev": 6, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "evexg": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "evexd": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "segadd": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "segex": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "segtrm": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "seg": 2, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "segmjr": 2, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "segexg": 0, 2024-03-21T09:11:23.505 INFO:teuthology.orchestra.run.smithi100.stdout: "segexd": 0, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "expos": 4194304, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "wrpos": 4195956, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "rdpos": 4195815, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "jlat": { 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "replayed": 6 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_mem": { 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "ino": 3, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "ino+": 3, 2024-03-21T09:11:23.506 INFO:teuthology.orchestra.run.smithi100.stdout: "ino-": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dir": 1, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dir+": 1, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dir-": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dn": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dn+": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dn-": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "cap": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "cap+": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "cap-": 0, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "rss": 48248, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "heap": 215420 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_server": { 2024-03-21T09:11:23.507 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_client_request": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_request": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_client_session": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "handle_peer_request": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "req_create_latency": { 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getattr_latency": { 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.508 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "req_link_latency": { 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookup_latency": { 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.509 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupino_latency": { 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.510 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupname_latency": { 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.511 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "req_lssnap_latency": { 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mkdir_latency": { 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.512 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mknod_latency": { 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: "req_mksnap_latency": { 2024-03-21T09:11:23.513 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "req_open_latency": { 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "req_readdir_latency": { 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.514 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rename_latency": { 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.515 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmdir_latency": { 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:23.516 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setattr_latency": { 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:23.517 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setlayout_latency": { 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "req_setxattr_latency": { 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.518 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "req_symlink_latency": { 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "req_unlink_latency": { 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:23.519 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_sessions": { 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "session_count": 0, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "session_add": 2, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "session_remove": 2, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_open": 1, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "sessions_stale": 0, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "total_load": 0, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "average_load": 0, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "avg_session_uptime": 1420, 2024-03-21T09:11:23.520 INFO:teuthology.orchestra.run.smithi100.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "mempool": { 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:23.521 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:23.522 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_items": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_bytes": 189463, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_anon_items": 52, 2024-03-21T09:11:23.523 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_bytes": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_items": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_items": 156, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "pgmap_items": 0, 2024-03-21T09:11:23.524 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_bytes": 8320, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "mds_co_items": 13, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_1_items": 0, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "unittest_2_items": 0 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "objecter": { 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_active": 0, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_laggy": 0, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send": 2771, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_send_bytes": 21, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_resend": 3, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_reply": 2771, 2024-03-21T09:11:23.525 INFO:teuthology.orchestra.run.smithi100.stdout: "op_latency": { 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2771, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 25.616355754, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.009244444 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "op_inflight": 0, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "oplen_avg": { 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 2771, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 2772 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "op": 2771, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "op_r": 2771, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "op_w": 0, 2024-03-21T09:11:23.526 INFO:teuthology.orchestra.run.smithi100.stdout: "op_rmw": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "op_pg": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_stat": 1816, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_create": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_read": 953, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_write": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writefull": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_writesame": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_append": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_zero": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_truncate": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_delete": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_mapext": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:23.527 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_getxattr": 1, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_setxattr": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_call": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_watch": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_notify": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "osdop_other": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_active": 0, 2024-03-21T09:11:23.528 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_send": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_resend": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "linger_ping": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_active": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_send": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "poolop_resend": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_active": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_send": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "poolstat_resend": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_active": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_send": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "statfs_resend": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "command_active": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "command_send": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "command_resend": 0, 2024-03-21T09:11:23.529 INFO:teuthology.orchestra.run.smithi100.stdout: "map_epoch": 88, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "map_full": 0, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "map_inc": 2, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_sessions": 5, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_open": 6, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_session_close": 1, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "osd_laggy": 0, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_wr": 0, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_rd": 2, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_del": 0 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "oft": { 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_objs": 0, 2024-03-21T09:11:23.530 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_updates": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "omap_total_removes": 0 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "osd-slow-ops": { 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "slow_ops_count": 0 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "purge_queue": { 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_executed": 0, 2024-03-21T09:11:23.531 INFO:teuthology.orchestra.run.smithi100.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 3055, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 1287196, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 3055, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 3055, 2024-03-21T09:11:23.532 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1287196, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 104857600, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:23.533 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 2771, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 1657, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2771, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 1657, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 1024, 2024-03-21T09:11:23.534 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 2771, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 2771, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 2771, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 2771, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.535 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:23.536 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: }, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "throttle-write_buf_throttle-0x5612e97c26e0": { 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "val": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "max": 3758096384, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "get_started": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "get": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "get_sum": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "take": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "take_sum": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "put": 0, 2024-03-21T09:11:23.537 INFO:teuthology.orchestra.run.smithi100.stdout: "put_sum": 0, 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout: "wait": { 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgcount": 0, 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout: "sum": 0.000000000, 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout: "avgtime": 0.000000000 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout: } 2024-03-21T09:11:23.538 INFO:teuthology.orchestra.run.smithi100.stdout:} 2024-03-21T09:11:24.304 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-03-21T09:11:24.304 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-03-21T09:11:24.304 DEBUG:tasks.check_counter:Getting stats from c 2024-03-21T09:11:24.304 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.c perf dump 2024-03-21T09:11:24.649 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:11:24.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:24 smithi105 ceph-mon[35993]: pgmap v780: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 31 MiB/s wr, 286 op/s 2024-03-21T09:11:25.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:24 smithi100 ceph-mon[36306]: pgmap v780: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 31 MiB/s wr, 286 op/s 2024-03-21T09:11:25.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:24 smithi087 ceph-mon[32385]: pgmap v780: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 31 MiB/s wr, 286 op/s 2024-03-21T09:11:26.808 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-21T09:11:26.808 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:26.808 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 2372, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 2374, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 428542266, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 754306, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.521847055, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.053883308, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.369824063, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.078926754, 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:26.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2372, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.148516604, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000062612 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.810 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1458, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1443, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 375600888, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 533480, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 6, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.416106453, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.042093296, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.316836243, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.048329819, 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:26.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1443, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.132905075, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000092103 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 822816, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 162944 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1297, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1499, 2024-03-21T09:11:26.812 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 382049423, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 933315, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 5, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.437492447, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.047969623, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.325171437, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.048883182, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1497, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.531764477, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000355220 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.813 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 320, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 521504 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.814 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7111, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.203442441, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000028609 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 0, 2024-03-21T09:11:26.816 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 36, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 4, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 32, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 29, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-21T09:11:26.817 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:26.818 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 4, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 234, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:26.819 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-21T09:11:26.820 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:26.821 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 645646, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-21T09:11:26.822 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 0, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 137, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 54, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 970357968, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 1188137442, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 1188136843, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.823 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 645646 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 23, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 36, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 13, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 9, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 21, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 12, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 36, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 57, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 21, 2024-03-21T09:11:26.824 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 54344, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 215420 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.825 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-21T09:11:26.826 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.827 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.828 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.829 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.830 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.830 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-21T09:11:26.830 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.830 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-21T09:11:26.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:26.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.835 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.836 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 3, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 2, 2024-03-21T09:11:26.837 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 3562, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:26.838 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:26.839 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:26.840 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 239004, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 85, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:26.841 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 88678, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 1400, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 4832, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 26, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 4, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 4832, 2024-03-21T09:11:26.842 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4832, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 40.360215914, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008352693 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4832, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 4833 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 4832, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 4832, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 0, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 3054, 2024-03-21T09:11:26.843 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 0, 2024-03-21T09:11:26.844 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 1775, 2024-03-21T09:11:26.844 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 0, 2024-03-21T09:11:26.844 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 0, 2024-03-21T09:11:26.844 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-21T09:11:26.844 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-21T09:11:26.844 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-21T09:11:26.845 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-21T09:11:26.845 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 0, 2024-03-21T09:11:26.845 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-21T09:11:26.845 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 2, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:26.846 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-21T09:11:26.847 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 11, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 11, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 0, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 2, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 0 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-21T09:11:26.848 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:26.849 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 5123, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1185713662, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 5123, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5123, 2024-03-21T09:11:26.850 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1185713662, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:26.851 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 4832, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1184058735, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4832, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1184058735, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:26.852 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 4832, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 4832, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4832, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 4832, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.853 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-21T09:11:26.854 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x561dea45a6e0": { 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:26.855 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:26.856 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-21T09:11:26.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:26 smithi105 ceph-mon[35993]: pgmap v781: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.3 MiB/s wr, 90 op/s 2024-03-21T09:11:27.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:26 smithi100 ceph-mon[36306]: pgmap v781: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.3 MiB/s wr, 90 op/s 2024-03-21T09:11:27.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:26 smithi087 ceph-mon[32385]: pgmap v781: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 7.3 MiB/s wr, 90 op/s 2024-03-21T09:11:27.604 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-03-21T09:11:27.605 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-03-21T09:11:27.605 DEBUG:tasks.check_counter:Getting stats from f 2024-03-21T09:11:27.605 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.f perf dump 2024-03-21T09:11:27.922 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:11:28.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:28 smithi105 ceph-mon[35993]: pgmap v782: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 56 op/s 2024-03-21T09:11:29.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:28 smithi100 ceph-mon[36306]: pgmap v782: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 56 op/s 2024-03-21T09:11:29.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:28 smithi087 ceph-mon[32385]: pgmap v782: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 5.2 MiB/s wr, 56 op/s 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 347, 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 335, 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 828726, 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 222138, 2024-03-21T09:11:29.789 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 14, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 1, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.055911440, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.014895115, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.034216850, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.007803133, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 334, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.074398828, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000222750 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 22, 2024-03-21T09:11:29.790 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000003018, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000137 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 824208, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 169936 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 20, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 222, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 7134, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 536008, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 9, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:29.791 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.012492648, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.007559335, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.003103550, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000327550, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 221, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.037857285, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000171299 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.792 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 523904 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 624, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 667, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 150111, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 148431, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 8, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.069295156, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.031208356, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.039800404, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.004850611, 2024-03-21T09:11:29.793 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 665, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.072998360, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000109771 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1229, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000357182, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000290 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:29.794 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 54, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001825090, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000033797 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.795 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000006966, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000003483 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 2, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 1, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-21T09:11:29.796 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000213673, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000213673 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 1, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 11, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 11, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:29.797 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 10, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 4, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 2, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 1, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-21T09:11:29.798 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 4, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 234, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:29.799 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:29.800 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 1 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 2, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:29.801 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.802 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 6, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 6, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-21T09:11:29.803 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4195956, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 5, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.009337966, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001867593 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 14, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 14, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-21T09:11:29.804 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 12, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 13, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 11, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 11, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 1, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 1, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 52132, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 223612 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 3, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:29.805 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 2, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 8, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 0, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000213673, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000213673 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:29.806 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:29.807 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.808 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.809 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.810 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.811 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.812 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.813 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.814 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.815 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.816 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 0, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-03-21T09:11:29.817 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 1, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 1, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 82, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:29.818 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:29.819 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:29.820 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 238279, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 72, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-21T09:11:29.821 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 54184, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 213, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-21T09:11:29.822 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 42, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 12044, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 33, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 42, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 42, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.223995526, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.005333226 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-21T09:11:29.823 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 42, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 64 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 42, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 42, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 5, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 8, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-21T09:11:29.824 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-21T09:11:29.825 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-21T09:11:29.826 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 8, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 13, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 5, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-21T09:11:29.827 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 14, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 1 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-21T09:11:29.828 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 987, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 877530, 2024-03-21T09:11:29.829 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 987, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 987, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 877530, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:29.830 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 42, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 12044, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 42, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 12044, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.831 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 42, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 42, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 42, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 42, 2024-03-21T09:11:29.832 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:29.833 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x5561a2336500": { 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:29.834 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 6, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1652, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 6, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 5, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1652, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:29.835 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-21T09:11:30.564 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-03-21T09:11:30.564 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-03-21T09:11:30.565 DEBUG:tasks.check_counter:Getting stats from i 2024-03-21T09:11:30.565 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.i perf dump 2024-03-21T09:11:30.956 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:30 smithi087 ceph-mon[32385]: pgmap v783: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 MiB/s wr, 190 op/s 2024-03-21T09:11:30.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:30 smithi105 ceph-mon[35993]: pgmap v783: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 MiB/s wr, 190 op/s 2024-03-21T09:11:31.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:30 smithi100 ceph-mon[36306]: pgmap v783: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 28 MiB/s wr, 190 op/s 2024-03-21T09:11:31.025 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:11:32.732 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:32 smithi105 ceph-mon[35993]: pgmap v784: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 30 MiB/s wr, 196 op/s 2024-03-21T09:11:32.981 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 6003, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 6411, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1702957, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 465370642, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 10, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 3.799378938, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 2.303836604, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.275626018, 2024-03-21T09:11:32.982 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 1.157372971, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 6415, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 3.707783308, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000577986 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3218, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001140099, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000354 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 0, 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 0 2024-03-21T09:11:32.983 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 4400, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 4560, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 1075310, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 616509205, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 11, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 3.420580075, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 2.303706220, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.159427371, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.910680285, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4558, 2024-03-21T09:11:32.984 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2.554341697, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000560408 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1190, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000370460, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000311 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 3344, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 528480 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 648304, 2024-03-21T09:11:32.985 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 650284, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 239007903, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 521889207, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 16, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 10, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 64.071749276, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 25.619818061, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 177.837179934, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 1.167544017, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 650278, 2024-03-21T09:11:32.986 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 310.293493721, 2024-03-21T09:11:32.989 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000477170 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1018825, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.530768444, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000520 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 238381696, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 259194208 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-21T09:11:32.990 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4671, 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 622.720428406, 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.133316298 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:32.991 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 476, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.016618098, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000034911 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 138, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 138, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 138, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 320.476388382, 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 2.322292669 2024-03-21T09:11:32.992 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 0, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 30, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 1, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 32, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 4, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 28, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 25, 2024-03-21T09:11:32.993 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 12, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 12, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 199, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 181, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-21T09:11:32.994 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 4, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 234, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:32.995 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 1, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 32, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 3170, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:32.996 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 644102, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 644025, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 3, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 64 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 0, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 8, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 9, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:32.997 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 22, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 13, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 1, 2024-03-21T09:11:32.998 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 645646, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 536621, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 528472, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 117174, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 8149, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 750, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 622, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 613, 2024-03-21T09:11:32.999 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 137, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 10, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 9, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 970357968, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 1188137442, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 7509, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 2091.790412828, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.278571103 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-21T09:11:33.000 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 32, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 45, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 13, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 18, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 30, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 12, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 32, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 66, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 34, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 12, 2024-03-21T09:11:33.001 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 44, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 32, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 89552, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 231804 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 191, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 1, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 138, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 22, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 2, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 22, 2024-03-21T09:11:33.002 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.022481337, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001021878 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 20, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 318.611632603, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 15.930581630 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.003 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 31, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 1.705972567, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.055031373 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.004 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000239878, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000119939 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.005 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.006 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.007 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.007 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-21T09:11:33.007 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10, 2024-03-21T09:11:33.007 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.015520002, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001552000 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.008 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 20, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.034136095, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001706804 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002765162, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001382581 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.009 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-21T09:11:33.010 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.034227573, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.008556893 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.011 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000987275, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000987275 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.012 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 8, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.018643819, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.002330477 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-21T09:11:33.013 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 18, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.029782071, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001654559 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.014 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 3, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 2, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 75, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 75, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 867, 2024-03-21T09:11:33.015 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:33.016 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:33.017 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 291524, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 126, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 0, 2024-03-21T09:11:33.018 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-21T09:11:33.019 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 124502, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 1811, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 10311, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 1397785424, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 21, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 10311, 2024-03-21T09:11:33.020 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10311, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 486.306506616, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.047163854 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 10311, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 10446 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 10311, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 2, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 10309, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-21T09:11:33.021 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 44, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 2, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 7511, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 88, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 1, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 1557, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-21T09:11:33.022 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 88, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-21T09:11:33.023 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-21T09:11:33.024 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 3, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 12, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 1142, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 13 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-21T09:11:33.025 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 6 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 513, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 6, 2024-03-21T09:11:33.026 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 533, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 13, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 658703, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 155377335, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 658703, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:33.027 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 658703, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 155377335, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:33.028 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 10311, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 1397786745, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 10311, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1397786745, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.029 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 10311, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 10311, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 10311, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 10311, 2024-03-21T09:11:33.030 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 13, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1321, 2024-03-21T09:11:33.031 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 13, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 2, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1321, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x564723782500": { 2024-03-21T09:11:33.032 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 645646, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1183943138, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 645646, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 7509, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1183943138, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:33.033 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.034 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:33.034 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-21T09:11:33.034 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:32 smithi100 ceph-mon[36306]: pgmap v784: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 30 MiB/s wr, 196 op/s 2024-03-21T09:11:33.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:32 smithi087 ceph-mon[32385]: pgmap v784: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 30 MiB/s wr, 196 op/s 2024-03-21T09:11:33.878 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-03-21T09:11:33.878 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-03-21T09:11:33.878 DEBUG:tasks.check_counter:Getting stats from l 2024-03-21T09:11:33.879 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11 shell --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 -- ceph daemon mds.l perf dump 2024-03-21T09:11:34.212 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/mon.c/config 2024-03-21T09:11:34.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:34 smithi105 ceph-mon[35993]: pgmap v785: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 49 MiB/s wr, 229 op/s 2024-03-21T09:11:35.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:34 smithi100 ceph-mon[36306]: pgmap v785: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 49 MiB/s wr, 229 op/s 2024-03-21T09:11:35.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:34 smithi087 ceph-mon[32385]: pgmap v785: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 49 MiB/s wr, 229 op/s 2024-03-21T09:11:36.084 INFO:teuthology.orchestra.run.smithi105.stdout:{ 2024-03-21T09:11:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-0": { 2024-03-21T09:11:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 1147, 2024-03-21T09:11:36.084 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 1040, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 996541, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 461780, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 3, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.115778853, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.045580013, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.068270158, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.012692874, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1039, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.131359375, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000126428 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.085 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1489, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000356983, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000239 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 808192, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 171664 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-1": { 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 28, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 234, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 8937, 2024-03-21T09:11:36.086 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 583833, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 16, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 7, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.017630236, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.008352766, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.007260458, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000316665, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 233, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.077526300, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000332730 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:36.087 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 21, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000009616, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000457 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 5856, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 529680 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "AsyncMessenger::Worker-2": { 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_messages": 34, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages": 36, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_bytes": 8033, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_bytes": 12012, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_created_connections": 7, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_active_connections": 4, 2024-03-21T09:11:36.088 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_total_time": 0.010292169, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_send_time": 0.002138363, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_recv_time": 0.007163330, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_running_fast_dispatch_time": 0.000538740, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_messages_queue_lat": { 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 34, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.017373026, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000510971 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_handle_ack_lat": { 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 25, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000006241, 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000249 2024-03-21T09:11:36.089 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_recv_encrypted_bytes": 2384, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "msgr_send_encrypted_bytes": 2160 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "cct": { 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "total_workers": 1, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "unhealthy_workers": 0 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-MDSRank": { 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 23, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002741109, 2024-03-21T09:11:36.090 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000119178 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "finisher-PurgeQueue": { 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "queue_len": 0, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "complete_latency": { 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000005649, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000002824 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "mds": { 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "request": 6, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "reply": 3, 2024-03-21T09:11:36.091 INFO:teuthology.orchestra.run.smithi105.stdout: "reply_latency": { 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 3, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.002821284, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000940428 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_reply": 0, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "forward": 3, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_complete": 0, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_fetch_keys": 0, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_commit": 11, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_split": 0, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_merge": 0, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes": 12, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_top": 0, 2024-03-21T09:11:36.092 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_bottom": 12, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pin_tail": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_pinned": 11, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_expired": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "inodes_with_caps": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "caps": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "subtrees": 2, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse": 8, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_hit": 6, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_forward": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_discover": 2, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_dir_fetch": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_remote_ino": 0, 2024-03-21T09:11:36.093 INFO:teuthology.orchestra.run.smithi105.stdout: "traverse_lock": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "load_cent": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "q": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "exported": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "exported_inodes": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "imported": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "imported_inodes": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_dir_fetch": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_backtrace_fetch": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "openino_peer_discover": 1, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rfiles": 4, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rbytes": 234, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "root_rsnaps": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_fetch": 0, 2024-03-21T09:11:36.094 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_set_tag": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_backtrace_repaired": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_inotable_repaired": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_inodes": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dir_base_inodes": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_dirfrag_rstats": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "scrub_file_inodes": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_inode_file_caps": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_revoke": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_grant": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_trunc": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "ceph_cap_op_flush_ack": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps": 0, 2024-03-21T09:11:36.095 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_caps_dirty": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_cap_release": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "process_request_cap_release": 0 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_cache": { 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_update_receipt": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_try_discover": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_send_discover": 3, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "dir_handle_discover": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_delayed": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "num_strays_enqueuing": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_created": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_enqueued": 0, 2024-03-21T09:11:36.096 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_reintegrated": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "strays_migrated": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_processing": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_enqueued": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "num_recovering_prioritized": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_started": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "recovery_completed": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_enqueue_scrub": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_exportdir": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_flush": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragmentdir": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_fragstats": 0, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "ireq_inodestats": 0 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.097 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_log": { 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "evlrg": 0, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "evadd": 5, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "evex": 0, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "evtrm": 0, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "ev": 5, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "evexg": 0, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "evexd": 0, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "segadd": 2, 2024-03-21T09:11:36.098 INFO:teuthology.orchestra.run.smithi105.stdout: "segex": 0, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "segtrm": 0, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "seg": 2, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "segmjr": 2, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "segexg": 0, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "segexd": 0, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "expos": 4194304, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "wrpos": 4195732, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "rdpos": 0, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "jlat": { 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 4, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.067210946, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.016802736 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.099 INFO:teuthology.orchestra.run.smithi105.stdout: "replayed": 0 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_mem": { 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "ino": 15, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "ino+": 15, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "ino-": 0, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "dir": 13, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "dir+": 14, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "dir-": 1, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "dn": 12, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "dn+": 12, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "dn-": 0, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "cap": 0, 2024-03-21T09:11:36.100 INFO:teuthology.orchestra.run.smithi105.stdout: "cap+": 3, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "cap-": 3, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "rss": 49776, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "heap": 223612 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_server": { 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_client_request": 11, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "dispatch_server_request": 0, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_request": 6, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_client_session": 16, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "handle_peer_request": 1, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "req_create_latency": { 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.101 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getattr_latency": { 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 1, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001692898, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.001692898 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getfilelock_latency": { 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.102 INFO:teuthology.orchestra.run.smithi105.stdout: "req_link_latency": { 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookup_latency": { 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 2, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.001128386, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000564193 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookuphash_latency": { 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.103 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupino_latency": { 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupname_latency": { 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupparent_latency": { 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.104 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lookupsnap_latency": { 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "req_lssnap_latency": { 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mkdir_latency": { 2024-03-21T09:11:36.105 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.106 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.106 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.106 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mknod_latency": { 2024-03-21T09:11:36.106 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "req_mksnap_latency": { 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "req_open_latency": { 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.107 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "req_readdir_latency": { 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rename_latency": { 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "req_renamesnap_latency": { 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.108 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_snapdiff_latency": { 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmdir_latency": { 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmsnap_latency": { 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.109 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "req_rmxattr_latency": { 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setattr_latency": { 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.110 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setdirlayout_latency": { 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setfilelock_latency": { 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setlayout_latency": { 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.111 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "req_setxattr_latency": { 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "req_symlink_latency": { 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "req_unlink_latency": { 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.112 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_revoke_eviction": 0, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "cap_acquisition_throttle": 0, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "req_getvxattr_latency": { 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_sessions": { 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "session_count": 1, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "session_add": 2, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "session_remove": 1, 2024-03-21T09:11:36.113 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_open": 1, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "sessions_stale": 0, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "total_load": 2, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "average_load": 2, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "avg_session_uptime": 867, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "mdthresh_evicted": 0 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "mempool": { 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_bytes": 0, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "bloom_filter_items": 0, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_bytes": 0, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_alloc_items": 0, 2024-03-21T09:11:36.114 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_data_items": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_onode_items": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_meta_items": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_other_items": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_cache_buffer_items": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_extent_items": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_bytes": 0, 2024-03-21T09:11:36.115 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_blob_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_shared_blob_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_inline_bl_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_fsck_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_txc_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_deferred_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluestore_writing_items": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_bytes": 0, 2024-03-21T09:11:36.116 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_items": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_bytes": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_reader_items": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_bytes": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "bluefs_file_writer_items": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_bytes": 248343, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_anon_items": 91, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_bytes": 176, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "buffer_meta_items": 2, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_bytes": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_items": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_bytes": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_mapbl_items": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_bytes": 0, 2024-03-21T09:11:36.117 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_pglog_items": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_bytes": 7432, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_items": 156, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_bytes": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "osdmap_mapping_items": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_bytes": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "pgmap_items": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_bytes": 58400, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "mds_co_items": 237, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_bytes": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_1_items": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_bytes": 0, 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: "unittest_2_items": 0 2024-03-21T09:11:36.118 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "objecter": { 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_active": 0, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_laggy": 0, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send": 42, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_send_bytes": 11955, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_resend": 33, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_reply": 42, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_latency": { 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 42, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.320993590, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.007642704 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "op_inflight": 0, 2024-03-21T09:11:36.119 INFO:teuthology.orchestra.run.smithi105.stdout: "oplen_avg": { 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 42, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 66 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "op": 42, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "op_r": 0, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "op_w": 42, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "op_rmw": 0, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "op_pg": 0, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_stat": 0, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_create": 10, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_read": 0, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_write": 4, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writefull": 7, 2024-03-21T09:11:36.120 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_writesame": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_append": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_zero": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_truncate": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_delete": 6, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_mapext": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_sparse_read": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_clonerange": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_getxattr": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_setxattr": 20, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_cmpxattr": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_rmxattr": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_resetxattrs": 0, 2024-03-21T09:11:36.121 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_call": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_watch": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_notify": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_src_cmpxattr": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_pgls_filter": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "osdop_other": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_active": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_send": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_resend": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "linger_ping": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_active": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_send": 0, 2024-03-21T09:11:36.122 INFO:teuthology.orchestra.run.smithi105.stdout: "poolop_resend": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_active": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_send": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "poolstat_resend": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_active": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_send": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "statfs_resend": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "command_active": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "command_send": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "command_resend": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "map_epoch": 88, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "map_full": 0, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "map_inc": 2, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_sessions": 9, 2024-03-21T09:11:36.123 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_open": 12, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_session_close": 3, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osd_laggy": 0, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_wr": 17, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_rd": 0, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_del": 2 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "oft": { 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_objs": 1, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_kv_pairs": 0, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_updates": 0, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "omap_total_removes": 0 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "osd-slow-ops": { 2024-03-21T09:11:36.124 INFO:teuthology.orchestra.run.smithi105.stdout: "slow_ops_count": 0 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "purge_queue": { 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_ops_high_water": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executing_high_water": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed_ops": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_executed": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "pq_item_in_journal": 0 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:36.125 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 1205, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 884727, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 1205, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 1205, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 884727, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.126 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_bytes": { 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 104857600, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 42, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 11955, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 42, 2024-03-21T09:11:36.127 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 11955, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-objecter_ops": { 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 1024, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:36.128 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 42, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 42, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 42, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 42, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle": { 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:36.129 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.130 INFO:teuthology.orchestra.run.smithi105.stdout: }, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "throttle-write_buf_throttle-0x55acb4982500": { 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "val": 0, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "max": 3758096384, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_started": 0, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get": 5, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_sum": 1428, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_fail": 0, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "get_or_fail_success": 5, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "take": 0, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "take_sum": 0, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "put": 4, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "put_sum": 1428, 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "wait": { 2024-03-21T09:11:36.131 INFO:teuthology.orchestra.run.smithi105.stdout: "avgcount": 0, 2024-03-21T09:11:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "sum": 0.000000000, 2024-03-21T09:11:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: "avgtime": 0.000000000 2024-03-21T09:11:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.132 INFO:teuthology.orchestra.run.smithi105.stdout: } 2024-03-21T09:11:36.132 INFO:teuthology.orchestra.run.smithi105.stdout:} 2024-03-21T09:11:36.896 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-03-21T09:11:36.896 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-03-21T09:11:36.896 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-03-21T09:11:36.906 INFO:tasks.kclient:Unmounting kernel clients... 2024-03-21T09:11:36.907 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T09:11:36.907 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:11:36.944 DEBUG:teuthology.orchestra.run.smithi087:> set -ex 2024-03-21T09:11:36.944 DEBUG:teuthology.orchestra.run.smithi087:> dd if=/proc/self/mounts of=/dev/stdout 2024-03-21T09:11:36.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:36 smithi105 ceph-mon[35993]: pgmap v786: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 56 MiB/s wr, 241 op/s 2024-03-21T09:11:37.003 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:37.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:36 smithi100 ceph-mon[36306]: pgmap v786: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 56 MiB/s wr, 241 op/s 2024-03-21T09:11:37.072 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:36 smithi087 ceph-mon[32385]: pgmap v786: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 56 MiB/s wr, 241 op/s 2024-03-21T09:11:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.162+0000 7fbd4ca81640 1 -- 172.21.15.87:0/719486068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 msgr2=0x7fbd48109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.162+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/719486068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48109b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbd40009ca0 tx=0x7fbd4002c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.157 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 -- 172.21.15.87:0/719486068 shutdown_connections 2024-03-21T09:11:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/719486068 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48109b00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/719486068 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd481028b0 0x7fbd48102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/719486068 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd481016c0 0x7fbd48101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 -- 172.21.15.87:0/719486068 >> 172.21.15.87:0/719486068 conn(0x7fbd480fcc50 msgr2=0x7fbd480ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 -- 172.21.15.87:0/719486068 shutdown_connections 2024-03-21T09:11:37.158 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 -- 172.21.15.87:0/719486068 wait complete. 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 Processor -- start 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.163+0000 7fbd4ca81640 1 -- start start 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd481016c0 0x7fbd48075450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd481028b0 0x7fbd48073a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48073fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4810d340 con 0x7fbd48103270 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4810d1c0 con 0x7fbd481016c0 2024-03-21T09:11:37.160 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4810d4c0 con 0x7fbd481028b0 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48073fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48073fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54396/0 (socket says 172.21.15.87:54396) 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 -- 172.21.15.87:0/1010116755 learned_addr learned my addr 172.21.15.87:0/1010116755 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd3ffff640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd481028b0 0x7fbd48073a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4659c640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd481016c0 0x7fbd48075450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 -- 172.21.15.87:0/1010116755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd481028b0 msgr2=0x7fbd48073a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd481028b0 0x7fbd48073a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 -- 172.21.15.87:0/1010116755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd481016c0 msgr2=0x7fbd48075450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd481016c0 0x7fbd48075450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 -- 172.21.15.87:0/1010116755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48074860 con 0x7fbd48103270 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd46d9d640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48073fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd40037040 tx=0x7fbd40032f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.161 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40041020 con 0x7fbd48103270 2024-03-21T09:11:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fbd40034cd0 con 0x7fbd48103270 2024-03-21T09:11:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd481a1a20 con 0x7fbd48103270 2024-03-21T09:11:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd40039040 con 0x7fbd48103270 2024-03-21T09:11:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.164+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd481a1f70 con 0x7fbd48103270 2024-03-21T09:11:37.162 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.165+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd481089e0 con 0x7fbd48103270 2024-03-21T09:11:37.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.166+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fbd40044020 con 0x7fbd48103270 2024-03-21T09:11:37.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.167+0000 7fbd3f7fe640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbd340a2f20 0x7fbd340a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.167+0000 7fbd4659c640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbd340a2f20 0x7fbd340a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.167+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fbd400f11e0 con 0x7fbd48103270 2024-03-21T09:11:37.163 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.168+0000 7fbd4659c640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbd340a2f20 0x7fbd340a53e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbd48102710 tx=0x7fbd38005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.166 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.172+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fbd4004b020 con 0x7fbd48103270 2024-03-21T09:11:37.342 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.348+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd480622a0 con 0x7fbd48103270 2024-03-21T09:11:37.344 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.350+0000 7fbd3f7fe640 1 -- 172.21.15.87:0/1010116755 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7fbd400be0d0 con 0x7fbd48103270 2024-03-21T09:11:37.344 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T09:11:37.345 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbd340a2f20 msgr2=0x7fbd340a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbd340a2f20 0x7fbd340a53e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fbd48102710 tx=0x7fbd38005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 msgr2=0x7fbd48073fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48073fd0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fbd40037040 tx=0x7fbd40032f40 comp rx=0 tx=0).stop 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 shutdown_connections 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fbd340a2f20 0x7fbd340a53e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fbd48103270 0x7fbd48073fd0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fbd481028b0 0x7fbd48073a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 --2- 172.21.15.87:0/1010116755 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fbd481016c0 0x7fbd48075450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.347 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 >> 172.21.15.87:0/1010116755 conn(0x7fbd480fcc50 msgr2=0x7fbd48105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 shutdown_connections 2024-03-21T09:11:37.348 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.352+0000 7fbd4ca81640 1 -- 172.21.15.87:0/1010116755 wait complete. 2024-03-21T09:11:37.358 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-21T08:50:27.107860+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-03-21T08:55:04.486296+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-03-21T08:55:05.756309+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-21T09:11:37.358 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-03-21T09:11:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 -- 172.21.15.87:0/470876506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 msgr2=0x7f1af81009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 --2- 172.21.15.87:0/470876506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af81009c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009860 tx=0x7f1ae802bb70 comp rx=0 tx=0).stop 2024-03-21T09:11:37.505 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 -- 172.21.15.87:0/470876506 shutdown_connections 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 --2- 172.21.15.87:0/470876506 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af81089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 --2- 172.21.15.87:0/470876506 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1af8101790 0x7f1af8101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 --2- 172.21.15.87:0/470876506 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af81009c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 -- 172.21.15.87:0/470876506 >> 172.21.15.87:0/470876506 conn(0x7f1af80fbb30 msgr2=0x7f1af80fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 -- 172.21.15.87:0/470876506 shutdown_connections 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 -- 172.21.15.87:0/470876506 wait complete. 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.509+0000 7f1afd059640 1 Processor -- start 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 -- start start 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af8196940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1af8101790 0x7f1af8196e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af819def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af810c200 con 0x7f1af8102150 2024-03-21T09:11:37.506 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af810c080 con 0x7f1af8101790 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1af810c380 con 0x7f1af81005a0 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af819def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af6d9d640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af8196940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1aeffff640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1af8101790 0x7f1af8196e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af819def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:54410/0 (socket says 172.21.15.87:54410) 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 -- 172.21.15.87:0/1114524374 learned_addr learned my addr 172.21.15.87:0/1114524374 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 -- 172.21.15.87:0/1114524374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 msgr2=0x7f1af8196940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af8196940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 -- 172.21.15.87:0/1114524374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1af8101790 msgr2=0x7f1af8196e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1af8101790 0x7f1af8196e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 -- 172.21.15.87:0/1114524374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1af819e560 con 0x7f1af8102150 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af6d9d640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af8196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af759e640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af819def0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1af000efc0 tx=0x7f1af000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.507 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af0003970 con 0x7f1af8102150 2024-03-21T09:11:37.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af819e850 con 0x7f1af8102150 2024-03-21T09:11:37.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f1af0004020 con 0x7f1af8102150 2024-03-21T09:11:37.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af000b9f0 con 0x7f1af8102150 2024-03-21T09:11:37.508 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.510+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1af819ed50 con 0x7f1af8102150 2024-03-21T09:11:37.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.511+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1af81079e0 con 0x7f1af8102150 2024-03-21T09:11:37.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.513+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f1af001d020 con 0x7f1af8102150 2024-03-21T09:11:37.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.513+0000 7f1af4d99640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1ae40a2f20 0x7f1ae40a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.514+0000 7f1af6d9d640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1ae40a2f20 0x7f1ae40a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.514+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f1af00cf840 con 0x7f1af8102150 2024-03-21T09:11:37.509 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.514+0000 7f1af6d9d640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1ae40a2f20 0x7f1ae40a53e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009830 tx=0x7f1ae802e040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.511 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.517+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f1af009c6a0 con 0x7f1af8102150 2024-03-21T09:11:37.688 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.693+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1af802d200 con 0x7f1af8102150 2024-03-21T09:11:37.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.693+0000 7f1af4d99640 1 -- 172.21.15.87:0/1114524374 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v88) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1af009bfa0 con 0x7f1af8102150 2024-03-21T09:11:37.689 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 1 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1ae40a2f20 msgr2=0x7f1ae40a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1ae40a2f20 0x7f1ae40a53e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8009830 tx=0x7f1ae802e040 comp rx=0 tx=0).stop 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 msgr2=0x7f1af819def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af819def0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1af000efc0 tx=0x7f1af000c800 comp rx=0 tx=0).stop 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 shutdown_connections 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f1ae40a2f20 0x7f1ae40a53e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.690 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f1af8102150 0x7f1af819def0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f1af8101790 0x7f1af8196e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 --2- 172.21.15.87:0/1114524374 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f1af81005a0 0x7f1af8196940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 >> 172.21.15.87:0/1114524374 conn(0x7f1af80fbb30 msgr2=0x7f1af8104bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 shutdown_connections 2024-03-21T09:11:37.691 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.695+0000 7f1afd059640 1 -- 172.21.15.87:0/1114524374 wait complete. 2024-03-21T09:11:37.701 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-03-21T09:11:37.808 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 -- 172.21.15.87:0/3204445814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 msgr2=0x7fa930109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 --2- 172.21.15.87:0/3204445814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa930109b00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fa928009ca0 tx=0x7fa92802c0f0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 -- 172.21.15.87:0/3204445814 shutdown_connections 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 --2- 172.21.15.87:0/3204445814 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa930109b00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 --2- 172.21.15.87:0/3204445814 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9301028b0 0x7fa930102d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 --2- 172.21.15.87:0/3204445814 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9301016c0 0x7fa930101ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 -- 172.21.15.87:0/3204445814 >> 172.21.15.87:0/3204445814 conn(0x7fa9300fcc30 msgr2=0x7fa9300ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 -- 172.21.15.87:0/3204445814 shutdown_connections 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 -- 172.21.15.87:0/3204445814 wait complete. 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.812+0000 7fa936a81640 1 Processor -- start 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 -- start start 2024-03-21T09:11:37.809 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9301016c0 0x7fa930197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9301028b0 0x7fa930197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa93019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93010d340 con 0x7fa930103270 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93010d1c0 con 0x7fa9301028b0 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93010d4c0 con 0x7fa9301016c0 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa93019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa93019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45460/0 (socket says 172.21.15.87:45460) 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 -- 172.21.15.87:0/3075920240 learned_addr learned my addr 172.21.15.87:0/3075920240 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa92ffff640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9301028b0 0x7fa930197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93481d640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9301016c0 0x7fa930197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 -- 172.21.15.87:0/3075920240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9301016c0 msgr2=0x7fa930197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9301016c0 0x7fa930197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 -- 172.21.15.87:0/3075920240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9301028b0 msgr2=0x7fa930197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.810 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9301028b0 0x7fa930197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 -- 172.21.15.87:0/3075920240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93019f570 con 0x7fa930103270 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa93501e640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa93019ef00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa92802c630 tx=0x7fa928032ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928041020 con 0x7fa930103270 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fa928034cd0 con 0x7fa930103270 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa93019f800 con 0x7fa930103270 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.813+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa928039040 con 0x7fa930103270 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.814+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa93019fd40 con 0x7fa930103270 2024-03-21T09:11:37.811 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.815+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9301089e0 con 0x7fa930103270 2024-03-21T09:11:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.816+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fa928044020 con 0x7fa930103270 2024-03-21T09:11:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.816+0000 7fa92dffb640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9200a2fa0 0x7fa9200a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.816+0000 7fa93481d640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9200a2fa0 0x7fa9200a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.817+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fa9280f1120 con 0x7fa930103270 2024-03-21T09:11:37.812 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.817+0000 7fa93481d640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9200a2fa0 0x7fa9200a5460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa930102710 tx=0x7fa924005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:37.815 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.821+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fa92804b020 con 0x7fa930103270 2024-03-21T09:11:37.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:37 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1010116755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:37.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:37 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1114524374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:37.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.995+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa9300622a0 con 0x7fa930103270 2024-03-21T09:11:37.990 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.996+0000 7fa92dffb640 1 -- 172.21.15.87:0/3075920240 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v88) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa9280be010 con 0x7fa930103270 2024-03-21T09:11:37.990 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 32 2024-03-21T09:11:37.992 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9200a2fa0 msgr2=0x7fa9200a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9200a2fa0 0x7fa9200a5460 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa930102710 tx=0x7fa924005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 msgr2=0x7fa93019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa93019ef00 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fa92802c630 tx=0x7fa928032ea0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 shutdown_connections 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fa9200a2fa0 0x7fa9200a5460 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fa930103270 0x7fa93019ef00 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fa9301028b0 0x7fa930197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 --2- 172.21.15.87:0/3075920240 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fa9301016c0 0x7fa930197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 >> 172.21.15.87:0/3075920240 conn(0x7fa9300fcc30 msgr2=0x7fa930105bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 shutdown_connections 2024-03-21T09:11:37.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:37.998+0000 7fa936a81640 1 -- 172.21.15.87:0/3075920240 wait complete. 2024-03-21T09:11:38.003 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:37 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1010116755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.003 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:37 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1114524374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:38.004 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-03-21T09:11:38.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:37 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1010116755' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:38.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:37 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1114524374' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-21T09:11:38.112 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 -- 172.21.15.87:0/3225908998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b101016d0 msgr2=0x7f2b10101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 --2- 172.21.15.87:0/3225908998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b101016d0 0x7f2b10101af0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f2b00009860 tx=0x7f2b0002bb70 comp rx=0 tx=0).stop 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 -- 172.21.15.87:0/3225908998 shutdown_connections 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 --2- 172.21.15.87:0/3225908998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b10103280 0x7f2b10109b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 --2- 172.21.15.87:0/3225908998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101028c0 0x7f2b10102d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 --2- 172.21.15.87:0/3225908998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b101016d0 0x7f2b10101af0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.115+0000 7f2b1481f640 1 -- 172.21.15.87:0/3225908998 >> 172.21.15.87:0/3225908998 conn(0x7f2b100fcc40 msgr2=0x7f2b100ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 -- 172.21.15.87:0/3225908998 shutdown_connections 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 -- 172.21.15.87:0/3225908998 wait complete. 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 Processor -- start 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 -- start start 2024-03-21T09:11:38.113 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 0x7f2b101979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b101028c0 0x7f2b10197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 0x7f2b1019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1010d320 con 0x7f2b10103280 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1010d1a0 con 0x7f2b101028c0 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b1481f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1010d4a0 con 0x7f2b101016d0 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.116+0000 7f2b0ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 0x7f2b1019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 0x7f2b101979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0ed9d640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 0x7f2b1019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45474/0 (socket says 172.21.15.87:45474) 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 0x7f2b101979e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.105:3300/0 says I am v2:172.21.15.87:40546/0 (socket says 172.21.15.87:40546) 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0ed9d640 1 -- 172.21.15.87:0/1108981064 learned_addr learned my addr 172.21.15.87:0/1108981064 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b07fff640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b101028c0 0x7f2b10197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 -- 172.21.15.87:0/1108981064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b101028c0 msgr2=0x7f2b10197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b101028c0 0x7f2b10197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.114 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 -- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 msgr2=0x7f2b1019ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 0x7f2b1019ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 -- 172.21.15.87:0/1108981064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1019f600 con 0x7f2b101016d0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0ed9d640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 0x7f2b1019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b0e59c640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 0x7f2b101979e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b0002c0b0 tx=0x7f2b00032430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00041020 con 0x7f2b101016d0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1019f890 con 0x7f2b101016d0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f2b0002c9f0 con 0x7f2b101016d0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b00039040 con 0x7f2b101016d0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.117+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1019fd80 con 0x7f2b101016d0 2024-03-21T09:11:38.115 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.118+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b101089f0 con 0x7f2b101016d0 2024-03-21T09:11:38.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.119+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f2b00044020 con 0x7f2b101016d0 2024-03-21T09:11:38.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.120+0000 7f2b077fe640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2afc0a2f20 0x7f2afc0a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.120+0000 7f2b07fff640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2afc0a2f20 0x7f2afc0a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.120+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f2b000f0d60 con 0x7f2b101016d0 2024-03-21T09:11:38.116 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.120+0000 7f2b07fff640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2afc0a2f20 0x7f2afc0a53e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2b10198ef0 tx=0x7f2af8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.118 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.124+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f2b0004b020 con 0x7f2b101016d0 2024-03-21T09:11:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.299+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2b100622a0 con 0x7f2b101016d0 2024-03-21T09:11:38.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.300+0000 7f2b077fe640 1 -- 172.21.15.87:0/1108981064 <== mon.1 v2:172.21.15.105:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v88) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2b000bdbc0 con 0x7f2b101016d0 2024-03-21T09:11:38.294 INFO:teuthology.orchestra.run.smithi087.stdout:pg_num: 64 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2afc0a2f20 msgr2=0x7f2afc0a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2afc0a2f20 0x7f2afc0a53e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2b10198ef0 tx=0x7f2af8009260 comp rx=0 tx=0).stop 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 msgr2=0x7f2b101979e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 0x7f2b101979e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f2b0002c0b0 tx=0x7f2b00032430 comp rx=0 tx=0).stop 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 shutdown_connections 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f2afc0a2f20 0x7f2afc0a53e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f2b10103280 0x7f2b1019ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f2b101028c0 0x7f2b10197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 --2- 172.21.15.87:0/1108981064 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f2b101016d0 0x7f2b101979e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 >> 172.21.15.87:0/1108981064 conn(0x7f2b100fcc40 msgr2=0x7f2b10105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 shutdown_connections 2024-03-21T09:11:38.297 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.302+0000 7f2b1481f640 1 -- 172.21.15.87:0/1108981064 wait complete. 2024-03-21T09:11:38.308 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/4153482495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f28103260 msgr2=0x7f3f28109af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/4153482495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f28103260 0x7f3f28109af0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3f20009ca0 tx=0x7f3f20034900 comp rx=0 tx=0).stop 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/4153482495 shutdown_connections 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/4153482495 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f28103260 0x7f3f28109af0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/4153482495 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f281028a0 0x7f3f28102d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/4153482495 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f281016b0 0x7f3f28101ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/4153482495 >> 172.21.15.87:0/4153482495 conn(0x7f3f280fcc40 msgr2=0x7f3f280ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/4153482495 shutdown_connections 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/4153482495 wait complete. 2024-03-21T09:11:38.413 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 Processor -- start 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- start start 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f281016b0 0x7f3f28195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 0x7f3f28195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f28103260 0x7f3f2819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2810b150 con 0x7f3f281028a0 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2810afd0 con 0x7f3f281016b0 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.416+0000 7f3f2ee4b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f2810b2d0 con 0x7f3f28103260 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 0x7f3f28195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 0x7f3f28195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45494/0 (socket says 172.21.15.87:45494) 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 -- 172.21.15.87:0/1238385599 learned_addr learned my addr 172.21.15.87:0/1238385599 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f2d3e8640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f28103260 0x7f3f2819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f2cbe7640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f281016b0 0x7f3f28195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.414 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 -- 172.21.15.87:0/1238385599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f28103260 msgr2=0x7f3f2819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f28103260 0x7f3f2819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 -- 172.21.15.87:0/1238385599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f281016b0 msgr2=0x7f3f28195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f281016b0 0x7f3f28195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 -- 172.21.15.87:0/1238385599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f2819d380 con 0x7f3f281028a0 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f27fff640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 0x7f3f28195ca0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009a10 tx=0x7f3f14009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400cc60 con 0x7f3f281028a0 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f3f1400ce00 con 0x7f3f281028a0 2024-03-21T09:11:38.415 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14017660 con 0x7f3f281028a0 2024-03-21T09:11:38.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f2819d670 con 0x7f3f281028a0 2024-03-21T09:11:38.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.417+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f2819dbc0 con 0x7f3f281028a0 2024-03-21T09:11:38.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.418+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f281089d0 con 0x7f3f281028a0 2024-03-21T09:11:38.416 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.419+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3f14017800 con 0x7f3f281028a0 2024-03-21T09:11:38.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.420+0000 7f3f25ffb640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3f180a2fa0 0x7f3f180a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:38.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.420+0000 7f3f2cbe7640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3f180a2fa0 0x7f3f180a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:38.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.420+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f3f140cf1a0 con 0x7f3f281028a0 2024-03-21T09:11:38.417 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.420+0000 7f3f2cbe7640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3f180a2fa0 0x7f3f180a5460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3f28102700 tx=0x7f3f1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:38.418 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.424+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f3f1409b9b0 con 0x7f3f281028a0 2024-03-21T09:11:38.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.613+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3f2802d070 con 0x7f3f281028a0 2024-03-21T09:11:38.607 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.613+0000 7f3f25ffb640 1 -- 172.21.15.87:0/1238385599 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) v1 ==== 81+0+588 (secure 0 0 0) 0x7f3f1409fa50 con 0x7f3f281028a0 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/2393104803 2024-03-22T08:47:58.771455+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/2393104803 2024-03-22T08:47:58.771455+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/845087826 2024-03-22T08:47:58.771455+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/4284800912 2024-03-22T08:47:31.789576+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3681410836 2024-03-22T08:47:31.789576+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/215090592 2024-03-22T08:47:31.789576+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6800/277415650 2024-03-22T08:47:31.789576+0000 2024-03-21T09:11:38.608 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3180053718 2024-03-22T08:47:58.771455+0000 2024-03-21T09:11:38.609 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:6801/277415650 2024-03-22T08:47:31.789576+0000 2024-03-21T09:11:38.609 INFO:teuthology.orchestra.run.smithi087.stdout:172.21.15.87:0/3000671039 2024-03-22T08:47:58.771455+0000 2024-03-21T09:11:38.609 INFO:teuthology.orchestra.run.smithi087.stderr:listed 10 entries 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3f180a2fa0 msgr2=0x7f3f180a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3f180a2fa0 0x7f3f180a5460 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f3f28102700 tx=0x7f3f1c005f90 comp rx=0 tx=0).stop 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 msgr2=0x7f3f28195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 0x7f3f28195ca0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f3f14009a10 tx=0x7f3f14009ed0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 shutdown_connections 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f3f180a2fa0 0x7f3f180a5460 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f3f28103260 0x7f3f2819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f3f281028a0 0x7f3f28195ca0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 --2- 172.21.15.87:0/1238385599 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f3f281016b0 0x7f3f28195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 >> 172.21.15.87:0/1238385599 conn(0x7f3f280fcc40 msgr2=0x7f3f28105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 shutdown_connections 2024-03-21T09:11:38.610 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:38.615+0000 7f3f2ee4b640 1 -- 172.21.15.87:0/1238385599 wait complete. 2024-03-21T09:11:38.621 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-03-21T09:11:38.621 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:38.621 DEBUG:teuthology.orchestra.run.smithi087:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-03-21T09:11:38.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:38 smithi087 ceph-mon[32385]: pgmap v787: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 58 MiB/s wr, 242 op/s 2024-03-21T09:11:38.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:38 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3075920240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:38.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:38 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1108981064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:38.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:38 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1238385599' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:11:38.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:38 smithi105 ceph-mon[35993]: pgmap v787: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 58 MiB/s wr, 242 op/s 2024-03-21T09:11:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:38 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3075920240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:38 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1108981064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:38.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:38 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1238385599' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:11:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:38 smithi100 ceph-mon[36306]: pgmap v787: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 58 MiB/s wr, 242 op/s 2024-03-21T09:11:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:38 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3075920240' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:38 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1108981064' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-03-21T09:11:39.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:38 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1238385599' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-21T09:11:40.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi087.front.sepia.ceph.com 2024-03-21T09:11:40.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:40.566 DEBUG:teuthology.orchestra.run.smithi087:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-03-21T09:11:40.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:40.616 DEBUG:teuthology.orchestra.run.smithi087:> ip netns list 2024-03-21T09:11:40.687 INFO:teuthology.orchestra.run.smithi087.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-03-21T09:11:40.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:40.688 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-03-21T09:11:40.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-03-21T09:11:40.720 DEBUG:teuthology.orchestra.run.smithi087:> sudo ip link delete ceph-brx 2024-03-21T09:11:40.819 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:40 smithi087 ceph-mon[32385]: pgmap v788: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 250 op/s 2024-03-21T09:11:40.820 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-21T09:11:40.829 INFO:tasks.cephadm:Teardown begin 2024-03-21T09:11:40.829 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-03-21T09:11:40.829 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.983+0000 7f707673b640 1 -- 172.21.15.87:0/1789528954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70701016e0 msgr2=0x7f7070101b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.978 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.983+0000 7f707673b640 1 --2- 172.21.15.87:0/1789528954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70701016e0 0x7f7070101b00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f7060009900 tx=0x7f706002bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.983+0000 7f707673b640 1 -- 172.21.15.87:0/1789528954 shutdown_connections 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.983+0000 7f707673b640 1 --2- 172.21.15.87:0/1789528954 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7070103290 0x7f7070109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.983+0000 7f707673b640 1 --2- 172.21.15.87:0/1789528954 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070102d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.983+0000 7f707673b640 1 --2- 172.21.15.87:0/1789528954 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f70701016e0 0x7f7070101b00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.984+0000 7f707673b640 1 -- 172.21.15.87:0/1789528954 >> 172.21.15.87:0/1789528954 conn(0x7f70700fcc50 msgr2=0x7f70700ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.984+0000 7f707673b640 1 -- 172.21.15.87:0/1789528954 shutdown_connections 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.984+0000 7f707673b640 1 -- 172.21.15.87:0/1789528954 wait complete. 2024-03-21T09:11:40.979 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.985+0000 7f707673b640 1 Processor -- start 2024-03-21T09:11:40.980 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 -- start start 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70701016e0 0x7f7070197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7070103290 0x7f707019f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707010d400 con 0x7f7070103290 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707010d280 con 0x7f70701028d0 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.986+0000 7f707673b640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707010d580 con 0x7f70701016e0 2024-03-21T09:11:40.981 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706ffff640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70701016e0 0x7f7070197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:49392/0 (socket says 172.21.15.87:49392) 2024-03-21T09:11:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 -- 172.21.15.87:0/2324870955 learned_addr learned my addr 172.21.15.87:0/2324870955 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f7074cd8640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7070103290 0x7f707019f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.982 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 -- 172.21.15.87:0/2324870955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70701016e0 msgr2=0x7f7070197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70701016e0 0x7f7070197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 -- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7070103290 msgr2=0x7f707019f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7070103290 0x7f707019f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 -- 172.21.15.87:0/2324870955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707019f6c0 con 0x7f70701028d0 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706ffff640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70701016e0 0x7f7070197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.987+0000 7f706f7fe640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070197fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f706400e9f0 tx=0x7f706400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.988+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f706400cd80 con 0x7f70701028d0 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.988+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7064010910 con 0x7f70701028d0 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.988+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7064018990 con 0x7f70701028d0 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.988+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f707019f9b0 con 0x7f70701028d0 2024-03-21T09:11:40.983 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.988+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f707019fdf0 con 0x7f70701028d0 2024-03-21T09:11:40.984 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.990+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7070108a00 con 0x7f70701028d0 2024-03-21T09:11:40.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.991+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7064010ac0 con 0x7f70701028d0 2024-03-21T09:11:40.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.991+0000 7f706d7fa640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f70580a30f0 0x7f70580a55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:40.985 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.991+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f70640cf6d0 con 0x7f70701028d0 2024-03-21T09:11:40.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.991+0000 7f706ffff640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f70580a30f0 0x7f70580a55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:40.986 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.992+0000 7f706ffff640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f70580a30f0 0x7f70580a55b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f706002c170 tx=0x7f706002df90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:40.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:40 smithi105 ceph-mon[35993]: pgmap v788: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 250 op/s 2024-03-21T09:11:40.993 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:40.999+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f706409c4d0 con 0x7f70701028d0 2024-03-21T09:11:41.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:40 smithi100 ceph-mon[36306]: pgmap v788: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 57 MiB/s wr, 250 op/s 2024-03-21T09:11:41.323 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.329+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f70700622a0 con 0x7f70701028d0 2024-03-21T09:11:41.324 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.330+0000 7f706d7fa640 1 -- 172.21.15.87:0/2324870955 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11460 (secure 0 0 0) 0x7f706409bdd0 con 0x7f70701028d0 2024-03-21T09:11:41.325 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T09:11:41.325 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:57:05.605808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24403,24487,24425,24451,24482]},"id":1}]} 2024-03-21T09:11:41.325 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-03-21T09:11:41.327 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f70580a30f0 msgr2=0x7f70580a55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f70580a30f0 0x7f70580a55b0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f706002c170 tx=0x7f706002df90 comp rx=0 tx=0).stop 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 msgr2=0x7f7070197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070197fe0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f706400e9f0 tx=0x7f706400eeb0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 shutdown_connections 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f70580a30f0 0x7f70580a55b0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7070103290 0x7f707019f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f70701028d0 0x7f7070197fe0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 --2- 172.21.15.87:0/2324870955 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f70701016e0 0x7f7070197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 >> 172.21.15.87:0/2324870955 conn(0x7f70700fcc50 msgr2=0x7f7070105bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 shutdown_connections 2024-03-21T09:11:41.328 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.333+0000 7f707673b640 1 -- 172.21.15.87:0/2324870955 wait complete. 2024-03-21T09:11:41.345 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-03-21T09:11:41.496 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 -- 172.21.15.87:0/2565609392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c107230 msgr2=0x7fb51c109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2565609392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c107230 0x7fb51c109620 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb514009be0 tx=0x7fb51402bef0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 -- 172.21.15.87:0/2565609392 shutdown_connections 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2565609392 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c107230 0x7fb51c109620 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2565609392 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51c104900 0x7fb51c106cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2565609392 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c101fd0 0x7fb51c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 -- 172.21.15.87:0/2565609392 >> 172.21.15.87:0/2565609392 conn(0x7fb51c0fb980 msgr2=0x7fb51c0fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.497 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 -- 172.21.15.87:0/2565609392 shutdown_connections 2024-03-21T09:11:41.498 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.502+0000 7fb522c8f640 1 -- 172.21.15.87:0/2565609392 wait complete. 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.503+0000 7fb522c8f640 1 Processor -- start 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.503+0000 7fb522c8f640 1 -- start start 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb522c8f640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 0x7fb51c195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb522c8f640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c104900 0x7fb51c195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb522c8f640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51c107230 0x7fb51c19c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb522c8f640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c10afb0 con 0x7fb51c104900 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb522c8f640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c10ae30 con 0x7fb51c101fd0 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb522c8f640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb51c10b130 con 0x7fb51c107230 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb52148c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c104900 0x7fb51c195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 0x7fb51c195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb52148c640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c104900 0x7fb51c195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45528/0 (socket says 172.21.15.87:45528) 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb52148c640 1 -- 172.21.15.87:0/2532051053 learned_addr learned my addr 172.21.15.87:0/2532051053 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:41.499 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb52248e640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51c107230 0x7fb51c19c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 0x7fb51c195400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.100:3300/0 says I am v2:172.21.15.87:49406/0 (socket says 172.21.15.87:49406) 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 -- 172.21.15.87:0/2532051053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51c107230 msgr2=0x7fb51c19c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51c107230 0x7fb51c19c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 -- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c104900 msgr2=0x7fb51c195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c104900 0x7fb51c195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.504+0000 7fb521c8d640 1 -- 172.21.15.87:0/2532051053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb51c19d020 con 0x7fb51c101fd0 2024-03-21T09:11:41.500 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.505+0000 7fb521c8d640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 0x7fb51c195400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51800e710 tx=0x7fb51800ebd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.505+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb51800cab0 con 0x7fb51c101fd0 2024-03-21T09:11:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.505+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7fb51800cc50 con 0x7fb51c101fd0 2024-03-21T09:11:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.505+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb518018750 con 0x7fb51c101fd0 2024-03-21T09:11:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.505+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb51c19d310 con 0x7fb51c101fd0 2024-03-21T09:11:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.505+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb51c19d810 con 0x7fb51c101fd0 2024-03-21T09:11:41.501 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.506+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb51c0fd470 con 0x7fb51c101fd0 2024-03-21T09:11:41.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.508+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7fb518018b00 con 0x7fb51c101fd0 2024-03-21T09:11:41.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.508+0000 7fb512ffd640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb5080a2ff0 0x7fb5080a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:41.502 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.508+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7fb5180cf660 con 0x7fb51c101fd0 2024-03-21T09:11:41.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.508+0000 7fb52148c640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb5080a2ff0 0x7fb5080a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:41.503 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.509+0000 7fb52148c640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb5080a2ff0 0x7fb5080a54b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb51c196910 tx=0x7fb50c011040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:41.510 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.516+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7fb51809be70 con 0x7fb51c101fd0 2024-03-21T09:11:41.837 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.842+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb51c0622a0 con 0x7fb51c101fd0 2024-03-21T09:11:41.838 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.844+0000 7fb512ffd640 1 -- 172.21.15.87:0/2532051053 <== mon.2 v2:172.21.15.100:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+11460 (secure 0 0 0) 0x7fb51809ff10 con 0x7fb51c101fd0 2024-03-21T09:11:41.838 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T09:11:41.838 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14553,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6835/2095083606","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6834","nonce":2095083606},{"type":"v1","addr":"172.21.15.87:6835","nonce":2095083606}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":4},{"gid":14565,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.87:6837/4247247896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6836","nonce":4247247896},{"type":"v1","addr":"172.21.15.87:6837","nonce":4247247896}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-03-21T08:55:06.567291+0000","modified":"2024-03-21T08:57:05.605808+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24487,"mds_1":24482,"mds_2":24451,"mds_3":24425,"mds_4":24403},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14634":{"gid":14634,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6839/3694307942","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6838","nonce":3694307942},{"type":"v1","addr":"172.21.15.87:6839","nonce":3694307942}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_14646":{"gid":14646,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.87:6841/2007269086","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6840","nonce":2007269086},{"type":"v1","addr":"172.21.15.87:6841","nonce":2007269086}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24395":{"gid":24395,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6833/1637520984","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6832","nonce":1637520984},{"type":"v1","addr":"172.21.15.100:6833","nonce":1637520984}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24403":{"gid":24403,"name":"l","rank":4,"incarnation":23,"state":"up:active","state_seq":12,"addr":"172.21.15.105:6833/500134057","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6832","nonce":500134057},{"type":"v1","addr":"172.21.15.105:6833","nonce":500134057}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24425":{"gid":24425,"name":"e","rank":3,"incarnation":21,"state":"up:active","state_seq":9,"addr":"172.21.15.100:6835/2548903221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6834","nonce":2548903221},{"type":"v1","addr":"172.21.15.100:6835","nonce":2548903221}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24451":{"gid":24451,"name":"f","rank":2,"incarnation":19,"state":"up:active","state_seq":8,"addr":"172.21.15.105:6835/1031026542","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6834","nonce":1031026542},{"type":"v1","addr":"172.21.15.105:6835","nonce":1031026542}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24455":{"gid":24455,"name":"h","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.100:6837/2948361419","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6836","nonce":2948361419},{"type":"v1","addr":"172.21.15.100:6837","nonce":2948361419}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24469":{"gid":24469,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.105:6837/2656551527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6836","nonce":2656551527},{"type":"v1","addr":"172.21.15.105:6837","nonce":2656551527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24482":{"gid":24482,"name":"b","rank":1,"incarnation":17,"state":"up:active","state_seq":4,"addr":"172.21.15.100:6839/214101858","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6838","nonce":214101858},{"type":"v1","addr":"172.21.15.100:6839","nonce":214101858}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_24487":{"gid":24487,"name":"i","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.105:6839/892092628","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6838","nonce":892092628},{"type":"v1","addr":"172.21.15.105:6839","nonce":892092628}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24487,"qdb_cluster":[24403,24487,24425,24451,24482]},"id":1}]} 2024-03-21T09:11:41.839 INFO:teuthology.orchestra.run.smithi087.stderr:dumped fsmap epoch 26 2024-03-21T09:11:41.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb5080a2ff0 msgr2=0x7fb5080a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.841 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb5080a2ff0 0x7fb5080a54b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fb51c196910 tx=0x7fb50c011040 comp rx=0 tx=0).stop 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 msgr2=0x7fb51c195400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 0x7fb51c195400 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb51800e710 tx=0x7fb51800ebd0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 shutdown_connections 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7fb5080a2ff0 0x7fb5080a54b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7fb51c107230 0x7fb51c19c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7fb51c104900 0x7fb51c195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 --2- 172.21.15.87:0/2532051053 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7fb51c101fd0 0x7fb51c195400 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 >> 172.21.15.87:0/2532051053 conn(0x7fb51c0fb980 msgr2=0x7fb51c102910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 shutdown_connections 2024-03-21T09:11:41.842 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:41.847+0000 7fb522c8f640 1 -- 172.21.15.87:0/2532051053 wait complete. 2024-03-21T09:11:41.858 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-03-21T09:11:41.981 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:41 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2324870955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:42.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 -- 172.21.15.87:0/1349211549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c102940 msgr2=0x7f0b1c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 --2- 172.21.15.87:0/1349211549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c102940 0x7f0b1c102dc0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f0b0c009930 tx=0x7f0b0c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:42.001 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 -- 172.21.15.87:0/1349211549 shutdown_connections 2024-03-21T09:11:42.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 --2- 172.21.15.87:0/1349211549 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b1c103300 0x7f0b1c109b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 --2- 172.21.15.87:0/1349211549 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c102940 0x7f0b1c102dc0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 --2- 172.21.15.87:0/1349211549 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c0ff970 0x7f0b1c0ffd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.002 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.007+0000 7f0b24152640 1 -- 172.21.15.87:0/1349211549 >> 172.21.15.87:0/1349211549 conn(0x7f0b1c0fb2a0 msgr2=0x7f0b1c0fd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:42.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.008+0000 7f0b24152640 1 -- 172.21.15.87:0/1349211549 shutdown_connections 2024-03-21T09:11:42.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.008+0000 7f0b24152640 1 -- 172.21.15.87:0/1349211549 wait complete. 2024-03-21T09:11:42.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.008+0000 7f0b24152640 1 Processor -- start 2024-03-21T09:11:42.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 -- start start 2024-03-21T09:11:42.003 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c0ff970 0x7f0b1c1a22d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b1c102940 0x7f0b1c1a2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 0x7f0b1c1a9880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c06c7f0 con 0x7f0b1c103300 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c06c670 con 0x7f0b1c0ff970 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 0x7f0b1c1a9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b24152640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1c06c970 con 0x7f0b1c102940 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 0x7f0b1c1a9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45536/0 (socket says 172.21.15.87:45536) 2024-03-21T09:11:42.004 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 -- 172.21.15.87:0/1412556612 learned_addr learned my addr 172.21.15.87:0/1412556612 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 -- 172.21.15.87:0/1412556612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b1c102940 msgr2=0x7f0b1c1a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b1c102940 0x7f0b1c1a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 -- 172.21.15.87:0/1412556612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c0ff970 msgr2=0x7f0b1c1a22d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c0ff970 0x7f0b1c1a22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 -- 172.21.15.87:0/1412556612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1c1a9ef0 con 0x7f0b1c103300 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.009+0000 7f0b226ef640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 0x7f0b1c1a9880 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400eb10 tx=0x7f0b1400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:42.005 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.010+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14004b20 con 0x7f0b1c103300 2024-03-21T09:11:42.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.010+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b1c1aa1e0 con 0x7f0b1c103300 2024-03-21T09:11:42.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.010+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b1c1aa780 con 0x7f0b1c103300 2024-03-21T09:11:42.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.010+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f0b14004cc0 con 0x7f0b1c103300 2024-03-21T09:11:42.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.010+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14010760 con 0x7f0b1c103300 2024-03-21T09:11:42.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.011+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1c1aaaf0 con 0x7f0b1c103300 2024-03-21T09:11:42.006 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.012+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f0b14005020 con 0x7f0b1c103300 2024-03-21T09:11:42.007 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.013+0000 7f0b12ffd640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b080a2ff0 0x7f0b080a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.013+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f0b140cfb60 con 0x7f0b1c103300 2024-03-21T09:11:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.013+0000 7f0b21eee640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b080a2ff0 0x7f0b080a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.008 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.014+0000 7f0b21eee640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b080a2ff0 0x7f0b080a54b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c068600 tx=0x7f0b18006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:42.014 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.020+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f0b1409c9c0 con 0x7f0b1c103300 2024-03-21T09:11:42.014 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:41 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2324870955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:42.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:41 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2324870955' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:42.289 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.294+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0b1c1aacc0 con 0x7f0b1c103300 2024-03-21T09:11:42.290 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.296+0000 7f0b12ffd640 1 -- 172.21.15.87:0/1412556612 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) v1 ==== 74+0+22616 (secure 0 0 0) 0x7f0b1409c2c0 con 0x7f0b1c103300 2024-03-21T09:11:42.291 INFO:teuthology.orchestra.run.smithi087.stdout: 2024-03-21T09:11:42.291 INFO:teuthology.orchestra.run.smithi087.stdout:{"epoch":88,"fsid":"75a2f1f0-e75f-11ee-95cd-87774f69a715","created":"2024-03-21T08:47:04.720561+0000","modified":"2024-03-21T08:56:00.061159+0000","last_up_change":"2024-03-21T08:54:16.704967+0000","last_in_change":"2024-03-21T08:53:56.222310+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-21T08:50:27.107860+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-03-21T08:55:04.486296+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-03-21T08:55:05.756309+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"399009d1-377c-4c98-80d9-f0b22744ad40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6802","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6803","nonce":3363556070}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6804","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6805","nonce":3363556070}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6808","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6809","nonce":3363556070}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6806","nonce":3363556070},{"type":"v1","addr":"172.21.15.87:6807","nonce":3363556070}]},"public_addr":"172.21.15.87:6803/3363556070","cluster_addr":"172.21.15.87:6805/3363556070","heartbeat_back_addr":"172.21.15.87:6809/3363556070","heartbeat_front_addr":"172.21.15.87:6807/3363556070","state":["exists","up"]},{"osd":1,"uuid":"f2bd15ff-1d4e-454c-bf79-12e1f1e53fd6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6800","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6801","nonce":4136971528}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6802","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6803","nonce":4136971528}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6806","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6807","nonce":4136971528}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6804","nonce":4136971528},{"type":"v1","addr":"172.21.15.100:6805","nonce":4136971528}]},"public_addr":"172.21.15.100:6801/4136971528","cluster_addr":"172.21.15.100:6803/4136971528","heartbeat_back_addr":"172.21.15.100:6807/4136971528","heartbeat_front_addr":"172.21.15.100:6805/4136971528","state":["exists","up"]},{"osd":2,"uuid":"90e5dc9a-effc-4556-a9a4-b84e24ae3338","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6800","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6801","nonce":3040563214}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6802","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6803","nonce":3040563214}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6806","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6807","nonce":3040563214}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6804","nonce":3040563214},{"type":"v1","addr":"172.21.15.105:6805","nonce":3040563214}]},"public_addr":"172.21.15.105:6801/3040563214","cluster_addr":"172.21.15.105:6803/3040563214","heartbeat_back_addr":"172.21.15.105:6807/3040563214","heartbeat_front_addr":"172.21.15.105:6805/3040563214","state":["exists","up"]},{"osd":3,"uuid":"bd94ec48-89ca-41e0-a0af-f91f7d8d3d14","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6810","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6811","nonce":269698093}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6812","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6813","nonce":269698093}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6816","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6817","nonce":269698093}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6814","nonce":269698093},{"type":"v1","addr":"172.21.15.87:6815","nonce":269698093}]},"public_addr":"172.21.15.87:6811/269698093","cluster_addr":"172.21.15.87:6813/269698093","heartbeat_back_addr":"172.21.15.87:6817/269698093","heartbeat_front_addr":"172.21.15.87:6815/269698093","state":["exists","up"]},{"osd":4,"uuid":"8e1e3ed0-cd0d-4ed4-9906-cfaf4a7f9e7d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6808","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6809","nonce":3040469961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6810","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6811","nonce":3040469961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6814","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6815","nonce":3040469961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6812","nonce":3040469961},{"type":"v1","addr":"172.21.15.100:6813","nonce":3040469961}]},"public_addr":"172.21.15.100:6809/3040469961","cluster_addr":"172.21.15.100:6811/3040469961","heartbeat_back_addr":"172.21.15.100:6815/3040469961","heartbeat_front_addr":"172.21.15.100:6813/3040469961","state":["exists","up"]},{"osd":5,"uuid":"4f908d67-699b-4d6f-8543-3f5dcaed7fcd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6808","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6809","nonce":1039939293}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6810","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6811","nonce":1039939293}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6814","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6815","nonce":1039939293}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6812","nonce":1039939293},{"type":"v1","addr":"172.21.15.105:6813","nonce":1039939293}]},"public_addr":"172.21.15.105:6809/1039939293","cluster_addr":"172.21.15.105:6811/1039939293","heartbeat_back_addr":"172.21.15.105:6815/1039939293","heartbeat_front_addr":"172.21.15.105:6813/1039939293","state":["exists","up"]},{"osd":6,"uuid":"fa673f89-d652-4f14-9f54-c034e053a748","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6818","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6819","nonce":944234078}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6820","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6821","nonce":944234078}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6824","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6825","nonce":944234078}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6822","nonce":944234078},{"type":"v1","addr":"172.21.15.87:6823","nonce":944234078}]},"public_addr":"172.21.15.87:6819/944234078","cluster_addr":"172.21.15.87:6821/944234078","heartbeat_back_addr":"172.21.15.87:6825/944234078","heartbeat_front_addr":"172.21.15.87:6823/944234078","state":["exists","up"]},{"osd":7,"uuid":"75a7a975-6491-4d37-b314-c75617e3b0cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":52,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6816","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6817","nonce":1198353584}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6818","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6819","nonce":1198353584}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6822","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6823","nonce":1198353584}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6820","nonce":1198353584},{"type":"v1","addr":"172.21.15.100:6821","nonce":1198353584}]},"public_addr":"172.21.15.100:6817/1198353584","cluster_addr":"172.21.15.100:6819/1198353584","heartbeat_back_addr":"172.21.15.100:6823/1198353584","heartbeat_front_addr":"172.21.15.100:6821/1198353584","state":["exists","up"]},{"osd":8,"uuid":"bd46697a-3872-410d-b1ab-63fa68278d79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6816","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6817","nonce":2122777536}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6818","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6819","nonce":2122777536}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6822","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6823","nonce":2122777536}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6820","nonce":2122777536},{"type":"v1","addr":"172.21.15.105:6821","nonce":2122777536}]},"public_addr":"172.21.15.105:6817/2122777536","cluster_addr":"172.21.15.105:6819/2122777536","heartbeat_back_addr":"172.21.15.105:6823/2122777536","heartbeat_front_addr":"172.21.15.105:6821/2122777536","state":["exists","up"]},{"osd":9,"uuid":"2aee04df-d0a2-48c3-89b5-eb79810633c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":68,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6826","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6827","nonce":3198171550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6828","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6829","nonce":3198171550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6832","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6833","nonce":3198171550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.87:6830","nonce":3198171550},{"type":"v1","addr":"172.21.15.87:6831","nonce":3198171550}]},"public_addr":"172.21.15.87:6827/3198171550","cluster_addr":"172.21.15.87:6829/3198171550","heartbeat_back_addr":"172.21.15.87:6833/3198171550","heartbeat_front_addr":"172.21.15.87:6831/3198171550","state":["exists","up"]},{"osd":10,"uuid":"cbfb1970-a260-4470-b56b-eff1363dd1b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":73,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6824","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6825","nonce":1731228246}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6826","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6827","nonce":1731228246}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6830","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6831","nonce":1731228246}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.100:6828","nonce":1731228246},{"type":"v1","addr":"172.21.15.100:6829","nonce":1731228246}]},"public_addr":"172.21.15.100:6825/1731228246","cluster_addr":"172.21.15.100:6827/1731228246","heartbeat_back_addr":"172.21.15.100:6831/1731228246","heartbeat_front_addr":"172.21.15.100:6829/1731228246","state":["exists","up"]},{"osd":11,"uuid":"d12fe1b9-2770-49d6-a7ca-d6595f7041f2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":81,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6824","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6825","nonce":3636544380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6826","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6827","nonce":3636544380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6830","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6831","nonce":3636544380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.105:6828","nonce":3636544380},{"type":"v1","addr":"172.21.15.105:6829","nonce":3636544380}]},"public_addr":"172.21.15.105:6825/3636544380","cluster_addr":"172.21.15.105:6827/3636544380","heartbeat_back_addr":"172.21.15.105:6831/3636544380","heartbeat_front_addr":"172.21.15.105:6829/3636544380","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:49:36.175400+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:00.415704+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:24.175898+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:50:49.633962+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:14.359876+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:51:39.068221+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:04.750540+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:30.289196+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:52:55.874765+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:22.329645+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:53:48.422281+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-21T08:54:14.841423+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.8","mappings":[{"from":2,"to":11}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"3.13","mappings":[{"from":4,"to":9}]},{"pgid":"3.14","mappings":[{"from":4,"to":6}]},{"pgid":"3.18","mappings":[{"from":3,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"3.37","mappings":[{"from":5,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.87:6801/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6800/2393104803":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/845087826":"2024-03-22T08:47:58.771455+0000","172.21.15.87:0/4284800912":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3681410836":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/215090592":"2024-03-22T08:47:31.789576+0000","172.21.15.87:6800/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3180053718":"2024-03-22T08:47:58.771455+0000","172.21.15.87:6801/277415650":"2024-03-22T08:47:31.789576+0000","172.21.15.87:0/3000671039":"2024-03-22T08:47:58.771455+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-21T09:11:42.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b080a2ff0 msgr2=0x7f0b080a54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b080a2ff0 0x7f0b080a54b0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c068600 tx=0x7f0b18006c90 comp rx=0 tx=0).stop 2024-03-21T09:11:42.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 msgr2=0x7f0b1c1a9880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 0x7f0b1c1a9880 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400eb10 tx=0x7f0b1400c2d0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.293 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 shutdown_connections 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f0b080a2ff0 0x7f0b080a54b0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f0b1c103300 0x7f0b1c1a9880 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f0b1c102940 0x7f0b1c1a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 --2- 172.21.15.87:0/1412556612 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f0b1c0ff970 0x7f0b1c1a22d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 >> 172.21.15.87:0/1412556612 conn(0x7f0b1c0fb2a0 msgr2=0x7f0b1c100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 shutdown_connections 2024-03-21T09:11:42.294 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.299+0000 7f0b24152640 1 -- 172.21.15.87:0/1412556612 wait complete. 2024-03-21T09:11:42.310 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-03-21T09:11:42.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.456+0000 7f5f25e30640 1 -- 172.21.15.87:0/3901253998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f201005e0 msgr2=0x7f5f20100a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.456+0000 7f5f25e30640 1 --2- 172.21.15.87:0/3901253998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f201005e0 0x7f5f20100a00 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009900 tx=0x7f5f1002bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:42.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 -- 172.21.15.87:0/3901253998 shutdown_connections 2024-03-21T09:11:42.451 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 --2- 172.21.15.87:0/3901253998 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f20102190 0x7f5f20108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 --2- 172.21.15.87:0/3901253998 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f201017d0 0x7f5f20101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 --2- 172.21.15.87:0/3901253998 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f201005e0 0x7f5f20100a00 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 -- 172.21.15.87:0/3901253998 >> 172.21.15.87:0/3901253998 conn(0x7f5f200fbb50 msgr2=0x7f5f200fdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:42.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 -- 172.21.15.87:0/3901253998 shutdown_connections 2024-03-21T09:11:42.452 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 -- 172.21.15.87:0/3901253998 wait complete. 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.457+0000 7f5f25e30640 1 Processor -- start 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 -- start start 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f201005e0 0x7f5f20110c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f201017d0 0x7f5f20111180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 0x7f5f201adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2010b220 con 0x7f5f20102190 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2010b0a0 con 0x7f5f201005e0 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f25e30640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2010b3a0 con 0x7f5f201017d0 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f1ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 0x7f5f201adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f1ffff640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 0x7f5f201adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45560/0 (socket says 172.21.15.87:45560) 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f1ffff640 1 -- 172.21.15.87:0/216960769 learned_addr learned my addr 172.21.15.87:0/216960769 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.458+0000 7f5f1f7fe640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f201005e0 0x7f5f20110c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.454 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1ffff640 1 -- 172.21.15.87:0/216960769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f201017d0 msgr2=0x7f5f20111180 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1ffff640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f201017d0 0x7f5f20111180 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1ffff640 1 -- 172.21.15.87:0/216960769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f201005e0 msgr2=0x7f5f20110c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1ffff640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f201005e0 0x7f5f20110c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1ffff640 1 -- 172.21.15.87:0/216960769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f201ae5b0 con 0x7f5f20102190 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1ffff640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 0x7f5f201adfc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b970 tx=0x7f5f1800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f18019070 con 0x7f5f20102190 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.459+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f201ae8a0 con 0x7f5f20102190 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.460+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f201aede0 con 0x7f5f20102190 2024-03-21T09:11:42.455 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.460+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f5f1800cb40 con 0x7f5f20102190 2024-03-21T09:11:42.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.460+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f1801d610 con 0x7f5f20102190 2024-03-21T09:11:42.456 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.461+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f0c0052c0 con 0x7f5f20102190 2024-03-21T09:11:42.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.463+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f5f1801c070 con 0x7f5f20102190 2024-03-21T09:11:42.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.463+0000 7f5f1cff9640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5f080a3000 0x7f5f080a54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.464+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8182+0+0 (secure 0 0 0) 0x7f5f180ce860 con 0x7f5f20102190 2024-03-21T09:11:42.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.464+0000 7f5f1f7fe640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5f080a3000 0x7f5f080a54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.458 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.464+0000 7f5f1f7fe640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5f080a3000 0x7f5f080a54c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009230 tx=0x7f5f10004f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:42.465 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.471+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f5f1809b010 con 0x7f5f20102190 2024-03-21T09:11:42.687 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.693+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5f0c0055b0 con 0x7f5f20102190 2024-03-21T09:11:42.689 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.695+0000 7f5f1cff9640 1 -- 172.21.15.87:0/216960769 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f1809f0b0 con 0x7f5f20102190 2024-03-21T09:11:42.689 INFO:teuthology.orchestra.run.smithi087.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-03-21T09:11:42.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5f080a3000 msgr2=0x7f5f080a54c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5f080a3000 0x7f5f080a54c0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f5f10009230 tx=0x7f5f10004f10 comp rx=0 tx=0).stop 2024-03-21T09:11:42.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 msgr2=0x7f5f201adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 0x7f5f201adfc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f5f1800b970 tx=0x7f5f1800be30 comp rx=0 tx=0).stop 2024-03-21T09:11:42.694 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 shutdown_connections 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f5f080a3000 0x7f5f080a54c0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f5f20102190 0x7f5f201adfc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f5f201017d0 0x7f5f20111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 --2- 172.21.15.87:0/216960769 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f5f201005e0 0x7f5f20110c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.699+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 >> 172.21.15.87:0/216960769 conn(0x7f5f200fbb50 msgr2=0x7f5f20104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.700+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 shutdown_connections 2024-03-21T09:11:42.695 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.700+0000 7f5f25e30640 1 -- 172.21.15.87:0/216960769 wait complete. 2024-03-21T09:11:42.707 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-03-21T09:11:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 -- 172.21.15.87:0/3803016793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 msgr2=0x7f17b0102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 --2- 172.21.15.87:0/3803016793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b0102340 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009900 tx=0x7f17ac02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 -- 172.21.15.87:0/3803016793 shutdown_connections 2024-03-21T09:11:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 --2- 172.21.15.87:0/3803016793 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17b0105230 0x7f17b01056b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.827 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 --2- 172.21.15.87:0/3803016793 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 0x7f17b0104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 --2- 172.21.15.87:0/3803016793 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b0102340 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.830+0000 7f17b73d4640 1 -- 172.21.15.87:0/3803016793 >> 172.21.15.87:0/3803016793 conn(0x7f17b00fbb30 msgr2=0x7f17b00fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 -- 172.21.15.87:0/3803016793 shutdown_connections 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 -- 172.21.15.87:0/3803016793 wait complete. 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 Processor -- start 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 -- start start 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b019df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 0x7f17b019e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17b0105230 0x7f17b01a5500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b01138f0 con 0x7f17b00fff10 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0113770 con 0x7f17b0102880 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b73d4640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17b0113a70 con 0x7f17b0105230 2024-03-21T09:11:42.828 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b5170640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b019df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b5170640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b019df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45602/0 (socket says 172.21.15.87:45602) 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b5170640 1 -- 172.21.15.87:0/2358370481 learned_addr learned my addr 172.21.15.87:0/2358370481 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.831+0000 7f17b496f640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 0x7f17b019e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5971640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17b0105230 0x7f17b01a5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5170640 1 -- 172.21.15.87:0/2358370481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17b0105230 msgr2=0x7f17b01a5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5170640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17b0105230 0x7f17b01a5500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5170640 1 -- 172.21.15.87:0/2358370481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 msgr2=0x7f17b019e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5170640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 0x7f17b019e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5170640 1 -- 172.21.15.87:0/2358370481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b01a5b70 con 0x7f17b00fff10 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b496f640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 0x7f17b019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b5170640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b019df50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009230 tx=0x7f17ac005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac041070 con 0x7f17b00fff10 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f17ac0044d0 con 0x7f17b00fff10 2024-03-21T09:11:42.829 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17b01a5e00 con 0x7f17b00fff10 2024-03-21T09:11:42.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17ac034440 con 0x7f17b00fff10 2024-03-21T09:11:42.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.832+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17b01a6260 con 0x7f17b00fff10 2024-03-21T09:11:42.830 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.833+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17b00fd620 con 0x7f17b00fff10 2024-03-21T09:11:42.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.834+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f17ac044070 con 0x7f17b00fff10 2024-03-21T09:11:42.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.834+0000 7f17a67fc640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f179c0a2fa0 0x7f179c0a5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:42.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.835+0000 7f17b496f640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f179c0a2fa0 0x7f179c0a5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:42.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.835+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8612+0+0 (secure 0 0 0) 0x7f17ac0f1150 con 0x7f17b00fff10 2024-03-21T09:11:42.831 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.835+0000 7f17b496f640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f179c0a2fa0 0x7f179c0a5460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f17b019f460 tx=0x7f17a000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:42.833 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:42.839+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f17ac0bde00 con 0x7f17b00fff10 2024-03-21T09:11:42.982 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c[66941]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:42.982 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c[66941]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:42.982 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mds[66945]: -- 172.21.15.105:0/2862989207 <== mon.1 v2:172.21.15.105:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55789693da40 con 0x557896910800 2024-03-21T09:11:42.982 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c[66941]: starting mds.c at 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mds[67249]: -- 172.21.15.105:0/1151064644 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a19f83da40 con 0x55a19f811000 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: starting mds.i at 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f[66632]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f[66632]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mds[66636]: -- 172.21.15.105:0/1537333696 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555d4b225a40 con 0x555d4b1f8c00 2024-03-21T09:11:42.983 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f[66632]: starting mds.f at 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: pgmap v789: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 46 MiB/s wr, 191 op/s 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2532051053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/1412556612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/216960769' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/216960769' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T09:11:42.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mon[35993]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T09:11:42.985 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l[66305]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:42.985 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l[66305]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:42.985 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-mds[66309]: -- 172.21.15.105:0/1612217197 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560bb183ba40 con 0x560bb180f000 2024-03-21T09:11:42.985 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:11:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l[66305]: starting mds.l at 2024-03-21T09:11:43.015 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b[66199]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:43.015 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b[66199]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:43.015 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mds[66203]: -- 172.21.15.100:0/3637287647 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56485f98fa40 con 0x56485f963000 2024-03-21T09:11:43.015 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b[66199]: starting mds.b at 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e[65573]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e[65573]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mds[65577]: -- 172.21.15.100:0/3157521493 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5633946c7a40 con 0x56339469a800 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e[65573]: starting mds.e at 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k[65254]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k[65254]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mds[65258]: -- 172.21.15.100:0/1029668402 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5637b79b1a40 con 0x5637b7984c00 2024-03-21T09:11:43.016 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k[65254]: starting mds.k at 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h[65893]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h[65893]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mds[65897]: -- 172.21.15.100:0/4107561719 <== mon.2 v2:172.21.15.100:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5597696b9a40 con 0x55976968d000 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h[65893]: starting mds.h at 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: pgmap v789: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 46 MiB/s wr, 191 op/s 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2532051053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/1412556612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:43.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/216960769' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:11:43.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T09:11:43.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:11:43.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/216960769' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T09:11:43.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T09:11:43.018 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:42 smithi100 ceph-mon[36306]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T09:11:43.049 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.055+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f17b0061ae0 con 0x7f17b00fff10 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a[73694]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a[73694]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mds[73698]: -- 172.21.15.87:0/3108044701 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55baf53d1a40 con 0x55baf53a4800 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a[73694]: starting mds.a at 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a[32381]: 2024-03-21T09:11:42.694+0000 7f5390089700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: pgmap v789: 97 pgs: 97 active+clean; 8.2 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 46 MiB/s wr, 191 op/s 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2532051053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-03-21T09:11:43.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/1412556612' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/216960769' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/216960769' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: osdmap e89: 12 total, 12 up, 12 in 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mon[32385]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g[73351]: ignoring --setuser ceph since I am not root 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g[73351]: ignoring --setgroup ceph since I am not root 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-mds[73355]: -- 172.21.15.87:0/2234640077 <== mon.0 v2:172.21.15.87:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55cc0e5e5a40 con 0x55cc0e5b8c00 2024-03-21T09:11:43.156 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:11:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g[73351]: starting mds.g at 2024-03-21T09:11:43.738 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.744+0000 7f17a67fc640 1 -- 172.21.15.87:0/2358370481 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17ac0bd700 con 0x7f17b00fff10 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f179c0a2fa0 msgr2=0x7f179c0a5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f179c0a2fa0 0x7f179c0a5460 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f17b019f460 tx=0x7f17a000a2d0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 msgr2=0x7f17b019df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b019df50 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009230 tx=0x7f17ac005d50 comp rx=0 tx=0).stop 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 shutdown_connections 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f179c0a2fa0 0x7f179c0a5460 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f17b0105230 0x7f17b01a5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f17b0102880 0x7f17b019e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 --2- 172.21.15.87:0/2358370481 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f17b00fff10 0x7f17b019df50 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.742 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 >> 172.21.15.87:0/2358370481 conn(0x7f17b00fbb30 msgr2=0x7f17b0103210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:43.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 shutdown_connections 2024-03-21T09:11:43.743 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.747+0000 7f17b73d4640 1 -- 172.21.15.87:0/2358370481 wait complete. 2024-03-21T09:11:43.760 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-03-21T09:11:43.873 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 -- 172.21.15.87:0/226362770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 msgr2=0x7f7f48104510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/226362770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48104510 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009900 tx=0x7f7f3c02bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 -- 172.21.15.87:0/226362770 shutdown_connections 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/226362770 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 0x7f7f481097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/226362770 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 0x7f7f48106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/226362770 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48104510 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 -- 172.21.15.87:0/226362770 >> 172.21.15.87:0/226362770 conn(0x7f7f480fbb10 msgr2=0x7f7f480fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 -- 172.21.15.87:0/226362770 shutdown_connections 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 -- 172.21.15.87:0/226362770 wait complete. 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 Processor -- start 2024-03-21T09:11:43.874 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.877+0000 7f7f4ed10640 1 -- start start 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 0x7f7f48197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 0x7f7f4819efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4810d340 con 0x7f7f48102120 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4810d1c0 con 0x7f7f48104a50 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4810d4c0 con 0x7f7f48107380 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45616/0 (socket says 172.21.15.87:45616) 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 -- 172.21.15.87:0/358898070 learned_addr learned my addr 172.21.15.87:0/358898070 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f47fff640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 0x7f7f48197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4d2ad640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 0x7f7f4819efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 -- 172.21.15.87:0/358898070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 msgr2=0x7f7f4819efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:43.875 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 0x7f7f4819efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 -- 172.21.15.87:0/358898070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 msgr2=0x7f7f48197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 0x7f7f48197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 -- 172.21.15.87:0/358898070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4819f630 con 0x7f7f48102120 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4d2ad640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 0x7f7f4819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f47fff640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 0x7f7f48197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4caac640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48197a10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009230 tx=0x7f7f3c005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c041070 con 0x7f7f48102120 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f7f3c0054e0 con 0x7f7f48102120 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.878+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f4819f8c0 con 0x7f7f48102120 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.879+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c034440 con 0x7f7f48102120 2024-03-21T09:11:43.876 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.879+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f4819fdb0 con 0x7f7f48102120 2024-03-21T09:11:43.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.880+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f480fd600 con 0x7f7f48102120 2024-03-21T09:11:43.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.881+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f7f3c044070 con 0x7f7f48102120 2024-03-21T09:11:43.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.881+0000 7f7f45ffb640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:43.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.881+0000 7f7f47fff640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:43.877 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.881+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 8612+0+0 (secure 0 0 0) 0x7f7f3c0f16f0 con 0x7f7f48102120 2024-03-21T09:11:43.878 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.882+0000 7f7f47fff640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f340058c0 tx=0x7f7f34005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:43.879 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:43.885+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f7f3c0be3a0 con 0x7f7f48102120 2024-03-21T09:11:44.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:43 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2358370481' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:11:44.055 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.060+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7f4802d070 con 0x7f7f48102120 2024-03-21T09:11:44.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:43 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2358370481' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:11:44.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:43 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2358370481' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-03-21T09:11:44.749 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.755+0000 7f7f45ffb640 1 -- 172.21.15.87:0/358898070 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v90) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7f3c0bdca0 con 0x7f7f48102120 2024-03-21T09:11:44.749 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_metadata' removed 2024-03-21T09:11:44.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 msgr2=0x7f7f380a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f7f340058c0 tx=0x7f7f34005850 comp rx=0 tx=0).stop 2024-03-21T09:11:44.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 msgr2=0x7f7f48197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48197a10 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c009230 tx=0x7f7f3c005dd0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 shutdown_connections 2024-03-21T09:11:44.753 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f7f380a2f20 0x7f7f380a53e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f7f48107380 0x7f7f4819efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f7f48104a50 0x7f7f48197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 --2- 172.21.15.87:0/358898070 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f7f48102120 0x7f7f48197a10 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 >> 172.21.15.87:0/358898070 conn(0x7f7f480fbb10 msgr2=0x7f7f48102bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:44.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.759+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 shutdown_connections 2024-03-21T09:11:44.754 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.760+0000 7f7f4ed10640 1 -- 172.21.15.87:0/358898070 wait complete. 2024-03-21T09:11:44.769 DEBUG:teuthology.orchestra.run.smithi087:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 -- 172.21.15.87:0/1179186107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 msgr2=0x7f381c101c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 --2- 172.21.15.87:0/1179186107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c101c10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3810009900 tx=0x7f381002bc30 comp rx=0 tx=0).stop 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 -- 172.21.15.87:0/1179186107 shutdown_connections 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 --2- 172.21.15.87:0/1179186107 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 0x7f381c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 --2- 172.21.15.87:0/1179186107 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c101c10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 --2- 172.21.15.87:0/1179186107 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f381c1005a0 0x7f381c1009c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.886 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 -- 172.21.15.87:0/1179186107 >> 172.21.15.87:0/1179186107 conn(0x7f381c0fbb30 msgr2=0x7f381c0fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 -- 172.21.15.87:0/1179186107 shutdown_connections 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.889+0000 7f38209cd640 1 -- 172.21.15.87:0/1179186107 wait complete. 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 Processor -- start 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 -- start start 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 --2- >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f381c1005a0 0x7f381c1969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c196ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 --2- >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 0x7f381c19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 -- --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c10c310 con 0x7f381c101790 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 -- --> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c10c190 con 0x7f381c1005a0 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f38209cd640 1 -- --> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f381c10c490 con 0x7f381c102150 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f3819d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c196ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f3819d9b640 1 --2- >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c196ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.87:3300/0 says I am v2:172.21.15.87:45622/0 (socket says 172.21.15.87:45622) 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f3819d9b640 1 -- 172.21.15.87:0/3781205385 learned_addr learned my addr 172.21.15.87:0/3781205385 (peer_addr_for_me v2:172.21.15.87:0/0) 2024-03-21T09:11:44.887 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f381a59c640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f381c1005a0 0x7f381c1969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.890+0000 7f381ad9d640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 0x7f381c19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f3819d9b640 1 -- 172.21.15.87:0/3781205385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 msgr2=0x7f381c19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f3819d9b640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 0x7f381c19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f3819d9b640 1 -- 172.21.15.87:0/3781205385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f381c1005a0 msgr2=0x7f381c1969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f3819d9b640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f381c1005a0 0x7f381c1969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f3819d9b640 1 -- 172.21.15.87:0/3781205385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f381c19e5d0 con 0x7f381c101790 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f381ad9d640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 0x7f381c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f3819d9b640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c196ef0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3810009230 tx=0x7f3810004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3810041070 con 0x7f381c101790 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f381c19e860 con 0x7f381c101790 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 2 ==== config(27 keys) v1 ==== 1140+0+0 (secure 0 0 0) 0x7f38100054e0 con 0x7f381c101790 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38100346a0 con 0x7f381c101790 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.891+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f381c19ed50 con 0x7f381c101790 2024-03-21T09:11:44.888 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.892+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f381c107940 con 0x7f381c101790 2024-03-21T09:11:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.893+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 4 ==== mgrmap(e 20) v1 ==== 137854+0+0 (secure 0 0 0) 0x7f3810044070 con 0x7f381c101790 2024-03-21T09:11:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.894+0000 7f380b7fe640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f38040a2f20 0x7f38040a53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-21T09:11:44.889 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.894+0000 7f381a59c640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f38040a2f20 0x7f38040a53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-21T09:11:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.894+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8116+0+0 (secure 0 0 0) 0x7f38100f15b0 con 0x7f381c101790 2024-03-21T09:11:44.890 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.894+0000 7f381a59c640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f38040a2f20 0x7f38040a53e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f381c1015f0 tx=0x7f38140094f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-21T09:11:44.891 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:44.897+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186149 (secure 0 0 0) 0x7f38100be4a0 con 0x7f381c101790 2024-03-21T09:11:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: pgmap v791: 97 pgs: 97 active+clean; 2.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 115 MiB/s wr, 927 op/s 2024-03-21T09:11:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T09:11:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T09:11:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: Cluster is now healthy 2024-03-21T09:11:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/2358370481' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:11:45.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6837/3842259250,v1:172.21.15.100:6839/3842259250] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6832/3885487254,v1:172.21.15.100:6833/3885487254] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6834/2444291724,v1:172.21.15.100:6835/2444291724] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.100:6836/1818504396,v1:172.21.15.100:6838/1818504396] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.87:6838/2372331409,v1:172.21.15.87:6839/2372331409] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.87:6840/3448827728,v1:172.21.15.87:6841/3448827728] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6836/4144890952,v1:172.21.15.105:6838/4144890952] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6837/3922182657,v1:172.21.15.105:6839/3922182657] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6832/1851255132,v1:172.21.15.105:6834/1851255132] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: mds.? [v2:172.21.15.105:6833/3436527616,v1:172.21.15.105:6835/3436527616] up:boot 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: fsmap 12 up:standby 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:11:45.016 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:11:45.017 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:44 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/358898070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:11:45.069 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.074+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 --> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f381c02d230 con 0x7f381c101790 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: pgmap v791: 97 pgs: 97 active+clean; 2.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 115 MiB/s wr, 927 op/s 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: Cluster is now healthy 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/2358370481' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6837/3842259250,v1:172.21.15.100:6839/3842259250] up:boot 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6832/3885487254,v1:172.21.15.100:6833/3885487254] up:boot 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6834/2444291724,v1:172.21.15.100:6835/2444291724] up:boot 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.100:6836/1818504396,v1:172.21.15.100:6838/1818504396] up:boot 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.87:6838/2372331409,v1:172.21.15.87:6839/2372331409] up:boot 2024-03-21T09:11:45.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.87:6840/3448827728,v1:172.21.15.87:6841/3448827728] up:boot 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6836/4144890952,v1:172.21.15.105:6838/4144890952] up:boot 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6837/3922182657,v1:172.21.15.105:6839/3922182657] up:boot 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6832/1851255132,v1:172.21.15.105:6834/1851255132] up:boot 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: mds.? [v2:172.21.15.105:6833/3436527616,v1:172.21.15.105:6835/3436527616] up:boot 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: fsmap 12 up:standby 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:11:45.156 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:11:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:11:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:11:45.157 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:44 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/358898070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: pgmap v791: 97 pgs: 97 active+clean; 2.9 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 KiB/s rd, 115 MiB/s wr, 927 op/s 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: Cluster is now healthy 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/2358370481' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6837/3842259250,v1:172.21.15.100:6839/3842259250] up:boot 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6832/3885487254,v1:172.21.15.100:6833/3885487254] up:boot 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6834/2444291724,v1:172.21.15.100:6835/2444291724] up:boot 2024-03-21T09:11:45.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.100:6836/1818504396,v1:172.21.15.100:6838/1818504396] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.87:6838/2372331409,v1:172.21.15.87:6839/2372331409] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.87:6840/3448827728,v1:172.21.15.87:6841/3448827728] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6836/4144890952,v1:172.21.15.105:6838/4144890952] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6837/3922182657,v1:172.21.15.105:6839/3922182657] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6832/1851255132,v1:172.21.15.105:6834/1851255132] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: mds.? [v2:172.21.15.105:6833/3436527616,v1:172.21.15.105:6835/3436527616] up:boot 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: fsmap 12 up:standby 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-03-21T09:11:45.233 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-03-21T09:11:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-03-21T09:11:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='mgr.14152 172.21.15.87:0/3566907055' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-03-21T09:11:45.234 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:44 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/358898070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:11:45.757 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.762+0000 7f380b7fe640 1 -- 172.21.15.87:0/3781205385 <== mon.0 v2:172.21.15.87:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v91) v1 ==== 169+0+0 (secure 0 0 0) 0x7f381c02d230 con 0x7f381c101790 2024-03-21T09:11:45.757 INFO:teuthology.orchestra.run.smithi087.stderr:pool 'cephfs_data' removed 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f38040a2f20 msgr2=0x7f38040a53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f38040a2f20 0x7f38040a53e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f381c1015f0 tx=0x7f38140094f0 comp rx=0 tx=0).stop 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 msgr2=0x7f381c196ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c196ef0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3810009230 tx=0x7f3810004040 comp rx=0 tx=0).stop 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 shutdown_connections 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:6800/1005139006,v1:172.21.15.87:6801/1005139006] conn(0x7f38040a2f20 0x7f38040a53e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.105:3300/0,v1:172.21.15.105:6789/0] conn(0x7f381c102150 0x7f381c19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.87:3300/0,v1:172.21.15.87:6789/0] conn(0x7f381c101790 0x7f381c196ef0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 --2- 172.21.15.87:0/3781205385 >> [v2:172.21.15.100:3300/0,v1:172.21.15.100:6789/0] conn(0x7f381c1005a0 0x7f381c1969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 >> 172.21.15.87:0/3781205385 conn(0x7f381c0fbb30 msgr2=0x7f381c104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 shutdown_connections 2024-03-21T09:11:45.760 INFO:teuthology.orchestra.run.smithi087.stderr:2024-03-21T09:11:45.765+0000 7f38209cd640 1 -- 172.21.15.87:0/3781205385 wait complete. 2024-03-21T09:11:45.779 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:11:45.816 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:11:45.855 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:11:46.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:45 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/358898070' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:11:46.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:45 smithi100 ceph-mon[36306]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T09:11:46.015 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:45 smithi100 ceph-mon[36306]: from='client.? 172.21.15.87:0/3781205385' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:11:46.154 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:45 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/358898070' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:11:46.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:45 smithi087 ceph-mon[32385]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T09:11:46.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:45 smithi087 ceph-mon[32385]: from='client.? 172.21.15.87:0/3781205385' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:11:46.231 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:45 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/358898070' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-03-21T09:11:46.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:45 smithi105 ceph-mon[35993]: osdmap e90: 12 total, 12 up, 12 in 2024-03-21T09:11:46.232 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:45 smithi105 ceph-mon[35993]: from='client.? 172.21.15.87:0/3781205385' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-21T09:11:46.288 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-21T09:11:46.288 DEBUG:teuthology.orchestra.run.smithi087:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:11:46.323 DEBUG:teuthology.orchestra.run.smithi100:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:11:46.338 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-21T09:11:46.353 INFO:tasks.cephadm:Stopping all daemons... 2024-03-21T09:11:46.354 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-21T09:11:46.354 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a 2024-03-21T09:11:46.654 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:46 smithi087 systemd[1]: Stopping Ceph mon.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:47.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a[32381]: 2024-03-21T09:11:46.721+0000 7f5395894700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:11:47.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a[32381]: 2024-03-21T09:11:46.721+0000 7f5395894700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-21T09:11:47.155 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:46 smithi087 podman[88714]: 2024-03-21 09:11:46.895702758 +0000 UTC m=+0.300747435 container died b1c901b5acae9f9a9cb3758fd5d36c814c02e635c57c0dac22dc45dd8623a752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:11:47.904 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:47 smithi087 podman[88714]: 2024-03-21 09:11:47.698479935 +0000 UTC m=+1.103524612 container cleanup b1c901b5acae9f9a9cb3758fd5d36c814c02e635c57c0dac22dc45dd8623a752 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-03-21T09:11:47.905 INFO:journalctl@ceph.mon.a.smithi087.stdout:Mar 21 09:11:47 smithi087 bash[88714]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-a 2024-03-21T09:11:48.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.a.service' 2024-03-21T09:11:48.128 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:11:48.128 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-21T09:11:48.128 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-03-21T09:11:48.129 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.b 2024-03-21T09:11:48.456 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:48 smithi100 systemd[1]: Stopping Ceph mon.b for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:48.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-b[36302]: 2024-03-21T09:11:48.462+0000 7f30e3373700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:11:48.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-b[36302]: 2024-03-21T09:11:48.462+0000 7f30e3373700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-03-21T09:11:48.765 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:48 smithi100 podman[70924]: 2024-03-21 09:11:48.483221025 +0000 UTC m=+0.143121345 container died 37d20ded27fc871e3adba095de5d2d1b34fc132699d49ed22482009335f2d464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-b, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-21T09:11:49.467 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:49 smithi100 podman[70924]: 2024-03-21 09:11:49.181010678 +0000 UTC m=+0.840910983 container cleanup 37d20ded27fc871e3adba095de5d2d1b34fc132699d49ed22482009335f2d464 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T09:11:49.468 INFO:journalctl@ceph.mon.b.smithi100.stdout:Mar 21 09:11:49 smithi100 bash[70924]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-b 2024-03-21T09:11:49.576 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.b.service' 2024-03-21T09:11:49.659 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:11:49.659 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-03-21T09:11:49.659 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-03-21T09:11:49.660 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.c 2024-03-21T09:11:49.982 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:49 smithi105 systemd[1]: Stopping Ceph mon.c for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:50.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-c[35967]: 2024-03-21T09:11:50.006+0000 7fa3f26b1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:11:50.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-c[35967]: 2024-03-21T09:11:50.006+0000 7fa3f26b1700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-03-21T09:11:50.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:50 smithi105 podman[72541]: 2024-03-21 09:11:50.231823036 +0000 UTC m=+0.337447988 container died c3d2d1514f72f72e1208c7f14b3bdd5915689a857903c7ee0dbd4e9d61bc2b0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-c, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-21T09:11:50.937 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:50 smithi105 podman[72541]: 2024-03-21 09:11:50.641878737 +0000 UTC m=+0.747503697 container cleanup c3d2d1514f72f72e1208c7f14b3bdd5915689a857903c7ee0dbd4e9d61bc2b0f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-c, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:11:50.937 INFO:journalctl@ceph.mon.c.smithi105.stdout:Mar 21 09:11:50 smithi105 bash[72541]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mon-c 2024-03-21T09:11:51.099 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mon.c.service' 2024-03-21T09:11:51.153 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:11:51.153 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-03-21T09:11:51.153 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-21T09:11:51.153 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x 2024-03-21T09:11:51.404 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:51 smithi087 systemd[1]: Stopping Ceph mgr.x for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:51.798 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x[32618]: 2024-03-21T09:11:51.477+0000 7fbbf9bb2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:11:51.798 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:51 smithi087 podman[88836]: 2024-03-21 09:11:51.509368872 +0000 UTC m=+0.152621268 container died 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:11:52.154 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:51 smithi087 podman[88836]: 2024-03-21 09:11:51.912846364 +0000 UTC m=+0.556098751 container cleanup 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:11:52.155 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:51 smithi087 bash[88836]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x 2024-03-21T09:11:52.436 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service' 2024-03-21T09:11:52.461 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:52 smithi087 podman[88850]: 2024-03-21 09:11:52.198032384 +0000 UTC m=+0.684595055 container remove 9d5a6cfb7056e8b4baa0ec2cdebbfab3a1802a5e35f3da7892e83f1794cd8624 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:2edfee68ad7243bed422a049b6f5f58619300f11, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-x, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:11:52.461 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:52 smithi087 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service: Deactivated successfully. 2024-03-21T09:11:52.461 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:52 smithi087 systemd[1]: Stopped Ceph mgr.x for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T09:11:52.461 INFO:journalctl@ceph.mgr.x.smithi087.stdout:Mar 21 09:11:52 smithi087 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.x.service: Consumed 1min 28.088s CPU time. 2024-03-21T09:11:52.474 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:11:52.474 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-21T09:11:52.475 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-21T09:11:52.475 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.y 2024-03-21T09:11:52.765 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 09:11:52 smithi100 systemd[1]: Stopping Ceph mgr.y for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:53.135 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 09:11:52 smithi100 podman[71045]: 2024-03-21 09:11:52.850870211 +0000 UTC m=+0.174769413 container died 5457a075ab1dabcaf51be4a515ca8459d41afd0b22903456f13b3a8ac5be85a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:11:53.515 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 09:11:53 smithi100 podman[71045]: 2024-03-21 09:11:53.330637471 +0000 UTC m=+0.654536678 container cleanup 5457a075ab1dabcaf51be4a515ca8459d41afd0b22903456f13b3a8ac5be85a1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-03-21T09:11:53.515 INFO:journalctl@ceph.mgr.y.smithi100.stdout:Mar 21 09:11:53 smithi100 bash[71045]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-y 2024-03-21T09:11:53.754 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.y.service' 2024-03-21T09:11:53.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:11:53.794 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-21T09:11:53.794 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-03-21T09:11:53.794 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.z 2024-03-21T09:11:54.128 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 09:11:53 smithi105 systemd[1]: Stopping Ceph mgr.z for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:54.434 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 09:11:54 smithi105 podman[72673]: 2024-03-21 09:11:54.135099874 +0000 UTC m=+0.136222896 container died fbd2b271f54d99b4dfe217a24711fa82370d233319d261d8367a9a31e5a79372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T09:11:54.732 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 09:11:54 smithi105 podman[72673]: 2024-03-21 09:11:54.539631688 +0000 UTC m=+0.540754689 container cleanup fbd2b271f54d99b4dfe217a24711fa82370d233319d261d8367a9a31e5a79372 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T09:11:54.732 INFO:journalctl@ceph.mgr.z.smithi105.stdout:Mar 21 09:11:54 smithi105 bash[72673]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mgr-z 2024-03-21T09:11:55.113 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mgr.z.service' 2024-03-21T09:11:55.152 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:11:55.152 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-03-21T09:11:55.152 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-21T09:11:55.152 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.0 2024-03-21T09:11:55.478 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:11:55 smithi087 systemd[1]: Stopping Ceph osd.0 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:11:55.904 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:11:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0[46874]: 2024-03-21T09:11:55.483+0000 7febb3037700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:11:55.904 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:11:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0[46874]: 2024-03-21T09:11:55.483+0000 7febb3037700 -1 osd.0 91 *** Got signal Terminated *** 2024-03-21T09:11:55.904 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:11:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0[46874]: 2024-03-21T09:11:55.483+0000 7febb3037700 -1 osd.0 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:01.155 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:00 smithi087 podman[88957]: 2024-03-21 09:12:00.857112786 +0000 UTC m=+5.493794100 container died 0b5e0a8f48596e9f3a9916de18eeaf572720dff857e33570548cf1cefad63090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:01.532 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:01 smithi087 podman[88957]: 2024-03-21 09:12:01.27035515 +0000 UTC m=+5.907036472 container cleanup 0b5e0a8f48596e9f3a9916de18eeaf572720dff857e33570548cf1cefad63090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-21T09:12:01.532 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:01 smithi087 bash[88957]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0 2024-03-21T09:12:01.829 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:01 smithi087 podman[88975]: 2024-03-21 09:12:01.538607731 +0000 UTC m=+0.667070670 container remove 0b5e0a8f48596e9f3a9916de18eeaf572720dff857e33570548cf1cefad63090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T09:12:02.145 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:01 smithi087 podman[89066]: 2024-03-21 09:12:01.738503271 +0000 UTC m=+0.017553427 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:02.146 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:01 smithi087 podman[89066]: 2024-03-21 09:12:01.869776938 +0000 UTC m=+0.148827128 container create 201750bdab3f3ad835357c9c42025ef5a62b46eeadb61a28a53606f3b90660f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:12:02.404 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:02 smithi087 podman[89066]: 2024-03-21 09:12:02.152360388 +0000 UTC m=+0.431410543 container init 201750bdab3f3ad835357c9c42025ef5a62b46eeadb61a28a53606f3b90660f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319) 2024-03-21T09:12:02.405 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:02 smithi087 podman[89066]: 2024-03-21 09:12:02.15609623 +0000 UTC m=+0.435146379 container start 201750bdab3f3ad835357c9c42025ef5a62b46eeadb61a28a53606f3b90660f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T09:12:02.405 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:02 smithi087 podman[89066]: 2024-03-21 09:12:02.309402964 +0000 UTC m=+0.588453114 container attach 201750bdab3f3ad835357c9c42025ef5a62b46eeadb61a28a53606f3b90660f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:12:04.382 INFO:journalctl@ceph.osd.0.smithi087.stdout:Mar 21 09:12:04 smithi087 podman[89066]: 2024-03-21 09:12:04.056191387 +0000 UTC m=+2.335241537 container died 201750bdab3f3ad835357c9c42025ef5a62b46eeadb61a28a53606f3b90660f2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-0-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:12:04.734 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.0.service' 2024-03-21T09:12:04.780 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:04.780 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-21T09:12:04.780 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-21T09:12:04.780 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.1 2024-03-21T09:12:05.104 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:04 smithi100 systemd[1]: Stopping Ceph osd.1 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:05.515 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1[40910]: 2024-03-21T09:12:05.109+0000 7efef942c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:05.515 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1[40910]: 2024-03-21T09:12:05.109+0000 7efef942c700 -1 osd.1 91 *** Got signal Terminated *** 2024-03-21T09:12:05.515 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1[40910]: 2024-03-21T09:12:05.109+0000 7efef942c700 -1 osd.1 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:10.765 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:10 smithi100 podman[71183]: 2024-03-21 09:12:10.5135174 +0000 UTC m=+5.534839478 container died 552a310e5bd893f5bf38ee9180d2476a799e37601c3226753fbcb5e943d95ec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-03-21T09:12:11.163 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:10 smithi100 podman[71183]: 2024-03-21 09:12:10.902128237 +0000 UTC m=+5.923450281 container cleanup 552a310e5bd893f5bf38ee9180d2476a799e37601c3226753fbcb5e943d95ec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:11.164 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:10 smithi100 bash[71183]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1 2024-03-21T09:12:11.469 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:11 smithi100 podman[71200]: 2024-03-21 09:12:11.170802964 +0000 UTC m=+0.653481132 container remove 552a310e5bd893f5bf38ee9180d2476a799e37601c3226753fbcb5e943d95ec2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-03-21T09:12:11.744 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:11 smithi100 podman[71292]: 2024-03-21 09:12:11.379170992 +0000 UTC m=+0.017387994 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:11.744 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:11 smithi100 podman[71292]: 2024-03-21 09:12:11.496638547 +0000 UTC m=+0.134855532 container create 48f04f033e5db24e632d1fd104ed68eb9b39a906c32bf89ac470f99b6d43bc1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:12:12.015 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:11 smithi100 podman[71292]: 2024-03-21 09:12:11.750848786 +0000 UTC m=+0.389065783 container init 48f04f033e5db24e632d1fd104ed68eb9b39a906c32bf89ac470f99b6d43bc1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T09:12:12.015 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:11 smithi100 podman[71292]: 2024-03-21 09:12:11.754332711 +0000 UTC m=+0.392549701 container start 48f04f033e5db24e632d1fd104ed68eb9b39a906c32bf89ac470f99b6d43bc1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T09:12:12.015 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:11 smithi100 podman[71292]: 2024-03-21 09:12:11.850864734 +0000 UTC m=+0.489081736 container attach 48f04f033e5db24e632d1fd104ed68eb9b39a906c32bf89ac470f99b6d43bc1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-03-21T09:12:12.383 INFO:journalctl@ceph.osd.1.smithi100.stdout:Mar 21 09:12:12 smithi100 podman[71292]: 2024-03-21 09:12:12.11655326 +0000 UTC m=+0.754770239 container died 48f04f033e5db24e632d1fd104ed68eb9b39a906c32bf89ac470f99b6d43bc1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-1-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-03-21T09:12:12.727 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.1.service' 2024-03-21T09:12:12.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:12.769 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-21T09:12:12.770 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-21T09:12:12.770 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.2 2024-03-21T09:12:13.088 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:12 smithi105 systemd[1]: Stopping Ceph osd.2 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:13.481 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2[41158]: 2024-03-21T09:12:13.094+0000 7fdd75ce8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:13.482 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2[41158]: 2024-03-21T09:12:13.094+0000 7fdd75ce8700 -1 osd.2 91 *** Got signal Terminated *** 2024-03-21T09:12:13.482 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2[41158]: 2024-03-21T09:12:13.094+0000 7fdd75ce8700 -1 osd.2 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:18.814 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:18 smithi105 podman[72794]: 2024-03-21 09:12:18.494081001 +0000 UTC m=+5.521884295 container died 2ab235da0c6d73966457210b26cd85697151d4a248187f3297741c655842122b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True) 2024-03-21T09:12:19.165 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:18 smithi105 podman[72794]: 2024-03-21 09:12:18.920317133 +0000 UTC m=+5.948120408 container cleanup 2ab235da0c6d73966457210b26cd85697151d4a248187f3297741c655842122b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-03-21T09:12:19.165 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:18 smithi105 bash[72794]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2 2024-03-21T09:12:19.481 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:19 smithi105 podman[72811]: 2024-03-21 09:12:19.171849712 +0000 UTC m=+0.679801781 container remove 2ab235da0c6d73966457210b26cd85697151d4a248187f3297741c655842122b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T09:12:19.833 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:19 smithi105 podman[72902]: 2024-03-21 09:12:19.484590666 +0000 UTC m=+0.017484752 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:19.833 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:19 smithi105 podman[72902]: 2024-03-21 09:12:19.614838295 +0000 UTC m=+0.147732380 container create 8dd003e8eaaed2fa69759703f07d72501d2dd3d90dffbe6de2d4e13b991d559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:12:19.833 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:19 smithi105 podman[72902]: 2024-03-21 09:12:19.839768051 +0000 UTC m=+0.372662146 container init 8dd003e8eaaed2fa69759703f07d72501d2dd3d90dffbe6de2d4e13b991d559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T09:12:20.232 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:19 smithi105 podman[72902]: 2024-03-21 09:12:19.843046921 +0000 UTC m=+0.375941007 container start 8dd003e8eaaed2fa69759703f07d72501d2dd3d90dffbe6de2d4e13b991d559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:20.232 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:19 smithi105 podman[72902]: 2024-03-21 09:12:19.951214706 +0000 UTC m=+0.484108901 container attach 8dd003e8eaaed2fa69759703f07d72501d2dd3d90dffbe6de2d4e13b991d559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2-deactivate, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-03-21T09:12:20.656 INFO:journalctl@ceph.osd.2.smithi105.stdout:Mar 21 09:12:20 smithi105 podman[72902]: 2024-03-21 09:12:20.406715121 +0000 UTC m=+0.939609208 container died 8dd003e8eaaed2fa69759703f07d72501d2dd3d90dffbe6de2d4e13b991d559a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319) 2024-03-21T09:12:21.009 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.2.service' 2024-03-21T09:12:21.051 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:21.051 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-21T09:12:21.051 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-21T09:12:21.051 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.3 2024-03-21T09:12:21.377 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:21 smithi087 systemd[1]: Stopping Ceph osd.3 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:21.654 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:21 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:21.382+0000 7f3570cf6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:21.654 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:21 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:21.382+0000 7f3570cf6700 -1 osd.3 91 *** Got signal Terminated *** 2024-03-21T09:12:21.655 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:21 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:21.382+0000 7f3570cf6700 -1 osd.3 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:22.981 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:22.553+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:23.128 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:22 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:22.728+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:23.404 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:23 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:23.133+0000 7f356d2ea700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.799251+0000 front 2024-03-21T09:11:58.799281+0000 (oldest deadline 2024-03-21T09:12:22.799952+0000) 2024-03-21T09:12:23.737 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:23 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:23.462+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:23.981 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:23.506+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:24.146 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:23 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:23.743+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:24.404 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:24 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:24.152+0000 7f356d2ea700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.799251+0000 front 2024-03-21T09:11:58.799281+0000 (oldest deadline 2024-03-21T09:12:22.799952+0000) 2024-03-21T09:12:24.731 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:24.470+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:24.779 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:24 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:24.446+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:25.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:24 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:24.784+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:25.467 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:25.185+0000 7f356d2ea700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.799251+0000 front 2024-03-21T09:11:58.799281+0000 (oldest deadline 2024-03-21T09:12:22.799952+0000) 2024-03-21T09:12:25.731 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:25.431+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:25.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:25.618+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:25.744 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:25.473+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:26.135 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:25 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:25.750+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:26.404 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3[53521]: 2024-03-21T09:12:26.141+0000 7f356d2ea700 -1 osd.3 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.799251+0000 front 2024-03-21T09:11:58.799281+0000 (oldest deadline 2024-03-21T09:12:22.799952+0000) 2024-03-21T09:12:26.711 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:26.432+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:26.731 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:26.619+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:26.731 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:26.446+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:27.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:26 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:26.674+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:27.023 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:26 smithi087 podman[89185]: 2024-03-21 09:12:26.751668453 +0000 UTC m=+5.501445550 container died 1f1819d7bedb36a3f086525980b4573200e1ed7f8dee6d14d92da7e1ac5acc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:12:27.024 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:26 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:26.716+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:27.404 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:27 smithi087 podman[89185]: 2024-03-21 09:12:27.133295529 +0000 UTC m=+5.883072639 container cleanup 1f1819d7bedb36a3f086525980b4573200e1ed7f8dee6d14d92da7e1ac5acc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-03-21T09:12:27.405 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:27 smithi087 bash[89185]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3 2024-03-21T09:12:27.727 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:27 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:27.471+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:27.727 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:27 smithi087 podman[89202]: 2024-03-21 09:12:27.443263988 +0000 UTC m=+0.684611810 container remove 1f1819d7bedb36a3f086525980b4573200e1ed7f8dee6d14d92da7e1ac5acc0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T09:12:27.731 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:27.633+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:27.731 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:27.465+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:28.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:27 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:27.722+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:28.040 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:27 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:27.754+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:28.040 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:27 smithi087 podman[89294]: 2024-03-21 09:12:27.636739463 +0000 UTC m=+0.017498885 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:28.040 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:27 smithi087 podman[89294]: 2024-03-21 09:12:27.769164804 +0000 UTC m=+0.149924259 container create 7d6ac545d5bd8915700d0371b9d3edd51ec4add847c77b173a67a84bc20d2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-03-21T09:12:28.040 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:28 smithi087 podman[89294]: 2024-03-21 09:12:28.046840166 +0000 UTC m=+0.427599593 container init 7d6ac545d5bd8915700d0371b9d3edd51ec4add847c77b173a67a84bc20d2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True) 2024-03-21T09:12:28.338 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:28 smithi087 podman[89294]: 2024-03-21 09:12:28.050479095 +0000 UTC m=+0.431238507 container start 7d6ac545d5bd8915700d0371b9d3edd51ec4add847c77b173a67a84bc20d2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:12:28.338 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:28 smithi087 podman[89294]: 2024-03-21 09:12:28.188957495 +0000 UTC m=+0.569716920 container attach 7d6ac545d5bd8915700d0371b9d3edd51ec4add847c77b173a67a84bc20d2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T09:12:28.613 INFO:journalctl@ceph.osd.3.smithi087.stdout:Mar 21 09:12:28 smithi087 podman[89294]: 2024-03-21 09:12:28.345253388 +0000 UTC m=+0.726012800 container died 7d6ac545d5bd8915700d0371b9d3edd51ec4add847c77b173a67a84bc20d2601 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240319) 2024-03-21T09:12:28.614 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:28 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:28.508+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:28.731 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:28.600+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:28.732 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:28.470+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:28.904 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:28 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:28.787+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:29.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:28 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:28.741+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:29.044 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.3.service' 2024-03-21T09:12:29.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:29.086 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-21T09:12:29.086 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-21T09:12:29.086 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.4 2024-03-21T09:12:29.394 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:29 smithi100 systemd[1]: Stopping Ceph osd.4 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:29.716 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:29 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4[47564]: 2024-03-21T09:12:29.399+0000 7ff40d883700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:29.716 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:29 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4[47564]: 2024-03-21T09:12:29.399+0000 7ff40d883700 -1 osd.4 91 *** Got signal Terminated *** 2024-03-21T09:12:29.716 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:29 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4[47564]: 2024-03-21T09:12:29.399+0000 7ff40d883700 -1 osd.4 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:29.734 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:29.483+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:29.734 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:29.587+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:29.788 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:29 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:29.478+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:30.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:29 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:29.722+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:30.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:29 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:29.794+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:30.732 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:30.470+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:30.601+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:30.773 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:30 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:30.435+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:31.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:30 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:30.757+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:31.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:30 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:30.779+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:31.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:30 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:30.779+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:31.812 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:31.449+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:31.812 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:31.449+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:31.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:31 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:31.591+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:31.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:31 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:31.509+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:31.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:31 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:31.509+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:32.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:31 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:31.723+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:32.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:31 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:31.692+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:32.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:31.818+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:32.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:31 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:31.818+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:32.830 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:32 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:32.477+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:32.830 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:32 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:32.477+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:32.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:32 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:32.511+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:32.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:32 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:32.511+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:32.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:32 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:32.605+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:32.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:32 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:32.605+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:33.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:32 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:32.660+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:33.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:32 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:32.752+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:33.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:32 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:32.752+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:33.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:32 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:32.836+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:33.155 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:32 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:32.836+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:33.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:33 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:33.232+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:33.808 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:33 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:33.464+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:33.808 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:33 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:33.464+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:33.981 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:33 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:33.631+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:33.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:33 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:33.631+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:33.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:33 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:33.533+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:33.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:33 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:33.533+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:34.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:33 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:33.657+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:34.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:33 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:33.777+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:34.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:33 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:33.777+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:34.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:33 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:33.814+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:34.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:33 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:33.814+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:34.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:34 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:34.199+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:34.857 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:34 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:34.465+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:34.858 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:34 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:34.465+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:34.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:34 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:34.572+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:34.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:34 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:34.572+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:34.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:34 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:34.597+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:34.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:34 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:34.597+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:35.015 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:34 smithi100 podman[71413]: 2024-03-21 09:12:34.921801938 +0000 UTC m=+5.635716563 container died 315ad875c64e8af5e5b5dd26b7edaf5c64aa4df7665da7e9d31b419459b1d6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2) 2024-03-21T09:12:35.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:34 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:34.684+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:35.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:34 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:34.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:35.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:34 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:34.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:35.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:34 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:34.864+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:35.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:34 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:34.864+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:35.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:35 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:35.158+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:35.514 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:35 smithi100 podman[71413]: 2024-03-21 09:12:35.309107175 +0000 UTC m=+6.023021768 container cleanup 315ad875c64e8af5e5b5dd26b7edaf5c64aa4df7665da7e9d31b419459b1d6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5) 2024-03-21T09:12:35.515 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:35 smithi100 bash[71413]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4 2024-03-21T09:12:35.863 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:35 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:35.475+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:35.863 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:35 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:35.475+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:35.918 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:35 smithi100 podman[71429]: 2024-03-21 09:12:35.619433257 +0000 UTC m=+0.692687767 container remove 315ad875c64e8af5e5b5dd26b7edaf5c64aa4df7665da7e9d31b419459b1d6d0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:12:35.919 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:35 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:35.673+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:35.919 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:35 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:35.769+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:35.919 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:35 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:35.769+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:35.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:35 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:35.570+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:35.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:35 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:35.570+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:35.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:35 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:35.621+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:35.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:35 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:35.621+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:36.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:35 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:35.869+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:36.155 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:35 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:35.869+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:36.170 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:35 smithi100 podman[71524]: 2024-03-21 09:12:35.827819287 +0000 UTC m=+0.017388585 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:36.170 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:35 smithi100 podman[71524]: 2024-03-21 09:12:35.962069085 +0000 UTC m=+0.151638412 container create e592a0111a49de82bd0463b81ccd9273f2dacb55ccdfb9fbf96ad0c5cf81e3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:36.170 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:36 smithi100 podman[71524]: 2024-03-21 09:12:36.173420802 +0000 UTC m=+0.362990089 container init e592a0111a49de82bd0463b81ccd9273f2dacb55ccdfb9fbf96ad0c5cf81e3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-03-21T09:12:36.170 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:36 smithi100 podman[71524]: 2024-03-21 09:12:36.176624569 +0000 UTC m=+0.366193856 container start e592a0111a49de82bd0463b81ccd9273f2dacb55ccdfb9fbf96ad0c5cf81e3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T09:12:36.460 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:36 smithi100 podman[71524]: 2024-03-21 09:12:36.272977081 +0000 UTC m=+0.462546377 container attach e592a0111a49de82bd0463b81ccd9273f2dacb55ccdfb9fbf96ad0c5cf81e3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-03-21T09:12:36.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:36 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:36.208+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:36.765 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:36 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:36.665+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:36.765 INFO:journalctl@ceph.osd.4.smithi100.stdout:Mar 21 09:12:36 smithi100 podman[71524]: 2024-03-21 09:12:36.467365504 +0000 UTC m=+0.656934794 container died e592a0111a49de82bd0463b81ccd9273f2dacb55ccdfb9fbf96ad0c5cf81e3bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-4-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:12:36.862 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:36.445+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:36.862 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:36.445+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:36.981 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:36 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:36.587+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:36.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:36 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:36.587+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:36.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:36 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:36.661+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:36.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:36 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:36.661+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:37.061 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:36 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:36.789+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:37.061 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:36 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:36.789+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:37.091 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.4.service' 2024-03-21T09:12:37.131 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:37.131 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-21T09:12:37.131 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-21T09:12:37.132 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.5 2024-03-21T09:12:37.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:36.868+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:37.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:36 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:36.868+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:37.435 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:37.211+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:37.435 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:37 smithi105 systemd[1]: Stopping Ceph osd.5 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:37.435 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:37.441+0000 7fa69f2f2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:37.435 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:37.441+0000 7fa69f2f2700 -1 osd.5 91 *** Got signal Terminated *** 2024-03-21T09:12:37.731 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:37.654+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:37.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:37.654+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:37.732 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:37.441+0000 7fa69f2f2700 -1 osd.5 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:37.732 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:37.547+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:37.732 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:37 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:37.547+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:37.903 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:37 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:37.423+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:37.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:37 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:37.423+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:38.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:37 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:37.674+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:38.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:37 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:37.795+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:38.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:37 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:37.795+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:38.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:37 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:37.910+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:38.154 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:37 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:37.910+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:38.574 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:38 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:38.253+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:38.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:38 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:38.458+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:38.905 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:38 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:38.458+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:38.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:38 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:38.614+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:38.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:38 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:38.614+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:38.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:38 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:38.580+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:38.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:38 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:38.580+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:39.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:38 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:38.660+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:39.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:38 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:38.750+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:39.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:38 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:38.750+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:39.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:38 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:38.946+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:39.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:38 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:38.946+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:39.561 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:39 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:39.247+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:39.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:39.422+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:39.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:39.422+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:39.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:39 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:39.565+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:39.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:39 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:39.565+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:39.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:39 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:39.661+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:39.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:39 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:39.661+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:40.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:39 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:39.705+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:40.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:39 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:39.705+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:40.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:39 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:39.705+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:40.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:39 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:39.671+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:40.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:39 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:39.671+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:40.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:39.955+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:40.405 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:39.955+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:40.405 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:39.955+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:40.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:40.212+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:40.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:40 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:40.448+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:40.905 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:40 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:40.448+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:40.981 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:40.544+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:40.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:40.544+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:40.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:40.705+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:40.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:40.705+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:40.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:40 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:40.705+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:41.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:40 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:40.630+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:41.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:40 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:40.630+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:41.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:40 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:40.741+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:41.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:40 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:40.741+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:41.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:40 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:40.741+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:41.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:40 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:40.978+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:41.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:40 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:40.978+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:41.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:40 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:40.978+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:41.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:41.213+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:41.866 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:41 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:41.412+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:41.867 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:41 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:41.412+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:41.981 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:41.750+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:41.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:41.750+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:41.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:41.750+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:41.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:41.506+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:56.565575+0000 front 2024-03-21T09:11:56.565393+0000 (oldest deadline 2024-03-21T09:12:21.865297+0000) 2024-03-21T09:12:41.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:41.506+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.867018+0000 front 2024-03-21T09:12:05.867194+0000 (oldest deadline 2024-03-21T09:12:31.166996+0000) 2024-03-21T09:12:41.982 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:41 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5[47705]: 2024-03-21T09:12:41.506+0000 7fa6978de700 -1 osd.5 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:16.268976+0000 front 2024-03-21T09:12:16.268995+0000 (oldest deadline 2024-03-21T09:12:40.969094+0000) 2024-03-21T09:12:42.017 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:41 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:41.632+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:42.018 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:41 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:41.632+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:42.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:41 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:41.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:42.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:41 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:41.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:42.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:41 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:41.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:42.155 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:42.018+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:42.155 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:42.018+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:42.155 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:42.018+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:42.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:42.182+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:42.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:42.433+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:42.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:42.433+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:42.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:42 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:42.433+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:43.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:42.676+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:43.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:42.676+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:43.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:42.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:43.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:42.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:43.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:42 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:42.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:43.166 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:42 smithi105 podman[73019]: 2024-03-21 09:12:42.918697133 +0000 UTC m=+5.593676537 container died 6d007d9684832ce21c5f27095afbb4ab2cb4a36ae39c6358548466619fb87096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319) 2024-03-21T09:12:43.166 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:42.769+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:43.166 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:42.769+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:43.167 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:42 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:42.769+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:43.386 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:43.047+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:43.387 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:43.047+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:43.387 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:43.047+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:43.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:43 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:43.172+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:43.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:43 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:43.172+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:43.482 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:43 smithi105 podman[73019]: 2024-03-21 09:12:43.267124148 +0000 UTC m=+5.942103527 container cleanup 6d007d9684832ce21c5f27095afbb4ab2cb4a36ae39c6358548466619fb87096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:43.482 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:43 smithi105 bash[73019]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5 2024-03-21T09:12:43.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:43.393+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:43.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:43.393+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:43.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:43.393+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:43.783 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:43 smithi105 podman[73036]: 2024-03-21 09:12:43.510181922 +0000 UTC m=+0.586277797 container remove 6d007d9684832ce21c5f27095afbb4ab2cb4a36ae39c6358548466619fb87096 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD) 2024-03-21T09:12:44.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:43 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:43.634+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:44.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:43 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:43.634+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:44.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:43 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:43.739+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:44.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:43 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:43.739+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:44.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:43 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:43.739+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:44.053 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:43 smithi105 podman[73125]: 2024-03-21 09:12:43.804049451 +0000 UTC m=+0.018222393 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:44.054 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:43 smithi105 podman[73125]: 2024-03-21 09:12:43.944754807 +0000 UTC m=+0.158927737 container create 39b88a00246df21a317c90016d60f24f82899d247f8aef28c6deff0121fc350f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2) 2024-03-21T09:12:44.054 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:43 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:43.788+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:44.054 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:43 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:43.789+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:44.054 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:43 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:43.789+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:44.360 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:44 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:44.050+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:44.360 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:44 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:44.050+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:44.360 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:44 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:44.050+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:44.471 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:44 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:44.160+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:44.472 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:44 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:44.160+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:44.472 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:44 smithi105 podman[73125]: 2024-03-21 09:12:44.1891839 +0000 UTC m=+0.403356844 container init 39b88a00246df21a317c90016d60f24f82899d247f8aef28c6deff0121fc350f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:44.472 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:44 smithi105 podman[73125]: 2024-03-21 09:12:44.192219441 +0000 UTC m=+0.406392380 container start 39b88a00246df21a317c90016d60f24f82899d247f8aef28c6deff0121fc350f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:44.472 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:44 smithi105 podman[73125]: 2024-03-21 09:12:44.272507632 +0000 UTC m=+0.486680572 container attach 39b88a00246df21a317c90016d60f24f82899d247f8aef28c6deff0121fc350f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-03-21T09:12:44.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:44 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:44.366+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:44.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:44 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:44.366+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:44.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:44 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:44.366+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:44.731 INFO:journalctl@ceph.osd.5.smithi105.stdout:Mar 21 09:12:44 smithi105 podman[73125]: 2024-03-21 09:12:44.478717839 +0000 UTC m=+0.692890779 container died 39b88a00246df21a317c90016d60f24f82899d247f8aef28c6deff0121fc350f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-03-21T09:12:45.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:44 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:44.589+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:45.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:44 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:44.589+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:45.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:44 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:44.701+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:45.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:44 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:44.701+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:45.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:44 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:44.701+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:45.086 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:44 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:44.811+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:45.086 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:44 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:44.811+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:45.086 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:44 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:44.811+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:45.116 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.5.service' 2024-03-21T09:12:45.157 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:45.158 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-21T09:12:45.158 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-21T09:12:45.158 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.6 2024-03-21T09:12:45.315 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:45.019+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:45.315 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:45.019+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:45.315 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:45.019+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:45.316 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 systemd[1]: Stopping Ceph osd.6 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:45.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:45 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:45.128+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:45.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:45 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:45.128+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:45.654 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:45.475+0000 7f5e6acef700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:45.655 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:45.475+0000 7f5e6acef700 -1 osd.6 91 *** Got signal Terminated *** 2024-03-21T09:12:45.655 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:45.475+0000 7f5e6acef700 -1 osd.6 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:45.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:45.320+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:45.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:45.320+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:45.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:45 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:45.320+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:46.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:45 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:45.621+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:46.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:45 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:45.621+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:46.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:45 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:45.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:46.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:45 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:45.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:46.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:45 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:45.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:46.097 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:45 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:45.797+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:46.097 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:45 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:45.797+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:46.097 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:45 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:45.797+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:46.347 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:46.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:46.347 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:46.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:46.348 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:46.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:46.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:46 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:46.102+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:46.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:46 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:46.102+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:46.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:46.353+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:46.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:46.353+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:46.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:46 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:46.353+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:47.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:46 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:46.643+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:47.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:46 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:46.643+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:47.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:46 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:46.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:47.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:46 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:46.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:47.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:46 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:46.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:47.127 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:46 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:46.843+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:47.128 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:46 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:46.843+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:47.128 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:46 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:46.843+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:47.388 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:47 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:47.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:47.388 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:47 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:47.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:47.388 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:47 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:47.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:47.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:47 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:47.134+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:47.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:47 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:47.134+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:47.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:47 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:47.394+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:47.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:47 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:47.394+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:47.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:47 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:47.394+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:48.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:47 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:47.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:48.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:47 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:47.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:48.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:47 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:47.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:48.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:47 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:47.667+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:48.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:47 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:47.667+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:48.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:47 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:47.667+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:48.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:48 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:48.124+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:48.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:48 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:48.124+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:48.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:48 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:48.124+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:48.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:47 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:47.883+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:48.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:47 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:47.883+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:48.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:47 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:47.883+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:48.348 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.023+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:48.349 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.023+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:48.349 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.023+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:48.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:48.355+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:48.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:48.355+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:48.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:48.355+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:48.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:48.355+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:49.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:48.711+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:49.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:48.711+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:49.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:48.711+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:49.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:48.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:49.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:48.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:49.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:48.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:49.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:48.754+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:49.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:49 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:49.077+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:49.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:49 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:49.077+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:49.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:49 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:49.077+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:49.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:48 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:48.929+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:49.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:48 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:48.929+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:49.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:48 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:48.929+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:49.384 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.995+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:49.384 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.995+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:49.385 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.995+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:49.385 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:48 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:48.995+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:25.232210+0000 front 2024-03-21T09:12:25.232244+0000 (oldest deadline 2024-03-21T09:12:48.732319+0000) 2024-03-21T09:12:49.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:49 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:49.391+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:49.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:49 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:49.391+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:49.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:49 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:49.391+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:49.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:49 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:49.391+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:49.749+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:49.749+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:49.749+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:49.749+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:49.747+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:49.747+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:50.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:49 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:49.747+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:50.231 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:49 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:49.929+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:50.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:49 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:49.929+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:50.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:49 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:49.929+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:50.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:50.058+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:50.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:50.058+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:50.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:50.058+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:50.348 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:50.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.327234+0000 front 2024-03-21T09:12:00.327290+0000 (oldest deadline 2024-03-21T09:12:22.627389+0000) 2024-03-21T09:12:50.348 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:50.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.829074+0000 front 2024-03-21T09:12:07.828974+0000 (oldest deadline 2024-03-21T09:12:30.629540+0000) 2024-03-21T09:12:50.348 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:50.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.631277+0000 front 2024-03-21T09:12:17.631358+0000 (oldest deadline 2024-03-21T09:12:39.331347+0000) 2024-03-21T09:12:50.348 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6[60319]: 2024-03-21T09:12:50.013+0000 7f5e632db700 -1 osd.6 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:25.232210+0000 front 2024-03-21T09:12:25.232244+0000 (oldest deadline 2024-03-21T09:12:48.732319+0000) 2024-03-21T09:12:50.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:50.354+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:50.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:50.354+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:50.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:50.354+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:50.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:50 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:50.354+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:51.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:50.704+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:51.042 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:50.704+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:51.042 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:50.704+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:51.042 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:50.713+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:51.042 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:50.713+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:51.042 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:50.713+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:51.042 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:50 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:50.713+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:51.096 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:50 smithi087 podman[89410]: 2024-03-21 09:12:50.824788345 +0000 UTC m=+5.464607329 container died 5414e0ba0aadcfd513f57397ddd478fac97d2f781258b9bcba0e5044de9b55ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0) 2024-03-21T09:12:51.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:51 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:51.022+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:51.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:51 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:51.022+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:51.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:51 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:51.022+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:51.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:50.940+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:51.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:50.940+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:51.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:50 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:50.940+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:51.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:51 smithi087 podman[89410]: 2024-03-21 09:12:51.202114488 +0000 UTC m=+5.841933482 container cleanup 5414e0ba0aadcfd513f57397ddd478fac97d2f781258b9bcba0e5044de9b55ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-03-21T09:12:51.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:51 smithi087 bash[89410]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6 2024-03-21T09:12:51.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:51.329+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:51.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:51.329+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:51.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:51.329+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:51.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:51 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:51.329+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:51.841 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:51 smithi087 podman[89429]: 2024-03-21 09:12:51.5371781 +0000 UTC m=+0.707586828 container remove 5414e0ba0aadcfd513f57397ddd478fac97d2f781258b9bcba0e5044de9b55ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:51.678+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:51.678+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:51.678+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:51.678+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:51.726+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:51.726+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:52.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:51.726+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:52.125 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:51 smithi087 podman[89519]: 2024-03-21 09:12:51.74860977 +0000 UTC m=+0.017591063 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:12:52.125 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:51 smithi087 podman[89519]: 2024-03-21 09:12:51.854607268 +0000 UTC m=+0.123588597 container create 2060726925ab6c0821e28bdc85aa1b9b9d52549cb67d6fac66253955b6f77dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True) 2024-03-21T09:12:52.125 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:52 smithi087 podman[89519]: 2024-03-21 09:12:52.057282466 +0000 UTC m=+0.326263750 container init 2060726925ab6c0821e28bdc85aa1b9b9d52549cb67d6fac66253955b6f77dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:12:52.126 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:52 smithi087 podman[89519]: 2024-03-21 09:12:52.060433004 +0000 UTC m=+0.329414291 container start 2060726925ab6c0821e28bdc85aa1b9b9d52549cb67d6fac66253955b6f77dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:12:52.126 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:52 smithi087 podman[89519]: 2024-03-21 09:12:52.13215132 +0000 UTC m=+0.401132616 container attach 2060726925ab6c0821e28bdc85aa1b9b9d52549cb67d6fac66253955b6f77dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T09:12:52.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:52 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:52.024+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:52.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:52 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:52.024+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:52.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:52 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:52.024+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:52.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:51 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:51.911+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:52.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:51 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:51.911+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:52.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:51 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:51.911+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:52.404 INFO:journalctl@ceph.osd.6.smithi087.stdout:Mar 21 09:12:52 smithi087 podman[89519]: 2024-03-21 09:12:52.356686755 +0000 UTC m=+0.625668043 container died 2060726925ab6c0821e28bdc85aa1b9b9d52549cb67d6fac66253955b6f77dd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-6-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-21T09:12:52.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:52 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:52.317+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:52.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:52 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:52.317+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:52.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:52 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:52.317+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:52.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:52 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:52.317+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:52.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:52.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:52.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:52.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:52.748+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:52.748+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:53.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:52 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:52.748+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:53.026 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.6.service' 2024-03-21T09:12:53.069 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:12:53.069 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-21T09:12:53.069 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-21T09:12:53.070 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.7 2024-03-21T09:12:53.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:53 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:53.055+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:53.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:53 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:53.055+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:53.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:53 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:53.055+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:53.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:52 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:52.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:53.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:52 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:52.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:53.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:52 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:52.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:53.389 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 systemd[1]: Stopping Ceph osd.7 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:12:53.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:53.319+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:53.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:53.319+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:53.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:53.319+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:53.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:53 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:53.319+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:53.692 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:53.395+0000 7f50a13bf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:12:53.693 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:53.395+0000 7f50a13bf700 -1 osd.7 91 *** Got signal Terminated *** 2024-03-21T09:12:53.693 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:53.395+0000 7f50a13bf700 -1 osd.7 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:12:53.693 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:53.699+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:53.693 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:53.699+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:53.693 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:53.699+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:54.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:53.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:54.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:53.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:54.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:53.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:54.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:53 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:53.742+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:54.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:54 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:54.036+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:54.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:54 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:54.036+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:54.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:54 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:54.036+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:54.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:53 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:53.933+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:54.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:53 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:53.933+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:54.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:53 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:53.933+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:54.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:54.273+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:54.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:54.273+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:54.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:54.273+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:54.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:54 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:54.273+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:54.711+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:54.711+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:54.711+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:54.703+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:54.703+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:54.703+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:55.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:54 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:54.703+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:55.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:55 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:55.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:55.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:55 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:55.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:55.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:55 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:55.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:55.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:54 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:54.978+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:55.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:54 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:54.978+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:55.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:54 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:54.978+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:55.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:55.226+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:55.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:55.226+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:55.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:55.226+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:55.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:55 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:55.226+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:56.018 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:55.677+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:56.018 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:55.677+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:56.019 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:55.677+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:56.019 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:55.720+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:56.019 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:55.720+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:56.019 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:55.720+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:56.019 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:55.720+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:56.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:56.076+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:56.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:56.076+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:56.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:56.076+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:56.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:55 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:55.978+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:56.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:55 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:55.978+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:56.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:55 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:55.978+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:56.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:56.222+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:56.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:56.222+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:56.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:56.222+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:56.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:56 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:56.222+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:56.665+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:56.665+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:56.665+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:56.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:56.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:56.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:57.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:56 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:56.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:57.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:57.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:57.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:57.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:57.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:56.963+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:56.963+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:56.963+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:57.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:56 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:56.963+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:12:57.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:57.213+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:57.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:57.213+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:57.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:57.213+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:57.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:57 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:57.213+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:58.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:57.669+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.428970+0000 front 2024-03-21T09:12:07.428948+0000 (oldest deadline 2024-03-21T09:12:31.529086+0000) 2024-03-21T09:12:58.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:57.669+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.032029+0000 front 2024-03-21T09:12:17.032065+0000 (oldest deadline 2024-03-21T09:12:39.332165+0000) 2024-03-21T09:12:58.015 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7[54315]: 2024-03-21T09:12:57.669+0000 7f509d9b3700 -1 osd.7 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:22.833370+0000 front 2024-03-21T09:12:22.833547+0000 (oldest deadline 2024-03-21T09:12:47.533303+0000) 2024-03-21T09:12:58.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:57.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:58.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:57.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:58.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:57.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:58.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:57 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:57.735+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:58.231 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:58.102+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:58.102+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:58.102+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:58.102+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:57.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:57.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:57.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:58.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:57 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:57.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:12:58.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:58.261+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:58.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:58.261+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:58.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:58.261+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:58.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:58 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:58.261+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:59.017 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:58 smithi100 podman[71629]: 2024-03-21 09:12:58.857680977 +0000 UTC m=+5.592395265 container died 7a71526d35403443b6ca86cda2c088f2486095c442fba310e9b6bedfec026036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:59.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:58 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:58.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:59.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:58 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:58.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:59.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:58 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:58.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:59.018 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:58 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:58.711+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:59.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:58.996+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:12:59.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:58.996+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:12:59.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:58.996+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:12:59.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:12:58 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:12:58.996+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:12:59.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:59 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:59.076+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:12:59.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:59 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:59.076+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:12:59.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:59 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:59.077+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:12:59.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:12:59 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:12:59.077+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:12:59.514 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:59 smithi100 podman[71629]: 2024-03-21 09:12:59.254123081 +0000 UTC m=+5.988837368 container cleanup 7a71526d35403443b6ca86cda2c088f2486095c442fba310e9b6bedfec026036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:12:59.515 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:59 smithi100 bash[71629]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7 2024-03-21T09:12:59.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:59 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:59.233+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:12:59.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:59 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:59.233+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:12:59.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:59 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:59.233+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:12:59.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:12:59 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:12:59.233+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:12:59.817 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:59 smithi100 podman[71658]: 2024-03-21 09:12:59.530489155 +0000 UTC m=+0.672125636 container remove 7a71526d35403443b6ca86cda2c088f2486095c442fba310e9b6bedfec026036 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T09:12:59.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:59 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:59.691+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:12:59.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:59 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:59.691+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:12:59.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:59 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:59.691+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:12:59.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:59 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:59.691+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:12:59.817 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:12:59 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:12:59.691+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:00.103 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:59 smithi100 podman[71749]: 2024-03-21 09:12:59.726688785 +0000 UTC m=+0.018006341 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:13:00.103 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:12:59 smithi100 podman[71749]: 2024-03-21 09:12:59.844328016 +0000 UTC m=+0.135645582 container create dd4d495563dfd8a40c62a80fffa3487280279df45cb11f8ab2627e6562cb4c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:00.103 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:13:00 smithi100 podman[71749]: 2024-03-21 09:13:00.109542688 +0000 UTC m=+0.400860245 container init dd4d495563dfd8a40c62a80fffa3487280279df45cb11f8ab2627e6562cb4c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T09:13:00.395 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:13:00 smithi100 podman[71749]: 2024-03-21 09:13:00.113210249 +0000 UTC m=+0.404527806 container start dd4d495563dfd8a40c62a80fffa3487280279df45cb11f8ab2627e6562cb4c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD) 2024-03-21T09:13:00.395 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:13:00 smithi100 podman[71749]: 2024-03-21 09:13:00.209193882 +0000 UTC m=+0.500511446 container attach dd4d495563dfd8a40c62a80fffa3487280279df45cb11f8ab2627e6562cb4c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T09:13:00.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:00.040+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:00.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:00.040+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:00.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:00.040+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:00.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:00.040+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:00.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:00.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:00.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:00.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:00.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:00.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:00.483 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:00 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:00.082+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:00.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:00 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:00.184+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:00.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:00 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:00.184+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:00.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:00 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:00.184+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:00.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:00 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:00.184+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:00.747 INFO:journalctl@ceph.osd.7.smithi100.stdout:Mar 21 09:13:00 smithi100 podman[71749]: 2024-03-21 09:13:00.402796226 +0000 UTC m=+0.694113788 container died dd4d495563dfd8a40c62a80fffa3487280279df45cb11f8ab2627e6562cb4c54 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-7-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-03-21T09:13:00.748 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:00.642+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:00.748 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:00.642+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:00.748 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:00.642+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:00.748 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:00.642+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:00.748 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:00.642+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:01.084 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.7.service' 2024-03-21T09:13:01.102 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:01.031+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:01.102 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:01.031+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:01.102 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:01.031+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:01.102 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:01.031+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:01.126 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:01.126 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-21T09:13:01.127 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-03-21T09:13:01.127 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.8 2024-03-21T09:13:01.379 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:01.379 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:01.379 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:01.379 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.109+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:01.379 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 systemd[1]: Stopping Ceph osd.8 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:01.556 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:01 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:01.223+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:01.556 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:01 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:01.223+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:01.556 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:01 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:01.223+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:01.556 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:01 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:01.223+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:01.731 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.454+0000 7fd7e7f1a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:01.731 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.454+0000 7fd7e7f1a700 -1 osd.8 91 *** Got signal Terminated *** 2024-03-21T09:13:01.732 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:01 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:01.454+0000 7fd7e7f1a700 -1 osd.8 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:13:02.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:01 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:01.654+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:02.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:01 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:01.654+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:02.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:01 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:01.654+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:02.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:01 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:01.654+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:02.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:01 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:01.654+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:02.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:02.118+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:02.119+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:02.119+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:02.119+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:02.024+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:02.024+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:02.024+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:02.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:02 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:02.024+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:02.654 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:02 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:02.204+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:02.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:02 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:02.204+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:02.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:02 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:02.204+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:02.655 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:02 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:02.204+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:03.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:02 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:02.696+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:03.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:02 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:02.696+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:03.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:02 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:02.696+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:03.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:02 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:02.696+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:03.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:02 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:02.696+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:03.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:03 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:03.160+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:03.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:03 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:03.160+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:03.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:03 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:03.160+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:03.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:03 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:03.160+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:03.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:03.110+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:03.110+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:03.110+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:03.110+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:03.044+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:03.044+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:03.044+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:03.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:03.044+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:04.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:03 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:03.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:04.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:03 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:03.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:04.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:03 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:03.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:04.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:03 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:03.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:04.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:03 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:03.714+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:04.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:04 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:04.148+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:04.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:04 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:04.148+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:04.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:04 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:04.148+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:04.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:04 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:04.148+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:04.118+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:04.118+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:04.118+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:04.118+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:04.118+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.170902+0000 front 2024-03-21T09:12:40.170967+0000 (oldest deadline 2024-03-21T09:13:03.671007+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:04.058+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:04.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:04.058+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:04.483 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:04.058+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:04.483 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:04.058+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:04.483 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:04 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:04.058+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:05.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:04 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:04.761+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:05.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:04 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:04.761+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:05.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:04 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:04.761+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:05.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:04 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:04.761+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:05.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:04 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:04.761+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:05.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:05 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:05.157+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:05.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:05 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:05.157+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:05.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:05 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:05.157+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:05.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:05 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:05.157+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:05.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:05.163+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:05.163+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:05.163+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:05.163+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:05.163+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.170902+0000 front 2024-03-21T09:12:40.170967+0000 (oldest deadline 2024-03-21T09:13:03.671007+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.028+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.028+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:05.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.028+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:05.483 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.028+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:05.483 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.028+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:06.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:05.732+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:06.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:05.732+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:06.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:05.732+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:06.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:05.732+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:06.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:05 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:05.732+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:06 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:06.193+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:08.363725+0000 front 2024-03-21T09:12:08.363875+0000 (oldest deadline 2024-03-21T09:12:33.063713+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:06 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:06.193+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.665483+0000 front 2024-03-21T09:12:17.665450+0000 (oldest deadline 2024-03-21T09:12:42.965647+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:06 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:06.193+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.967458+0000 front 2024-03-21T09:12:23.967574+0000 (oldest deadline 2024-03-21T09:12:48.067417+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:06 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:06.193+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.968687+0000 front 2024-03-21T09:12:30.968798+0000 (oldest deadline 2024-03-21T09:12:56.268780+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:06 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8[54595]: 2024-03-21T09:13:06.193+0000 7fd7e450e700 -1 osd.8 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.170902+0000 front 2024-03-21T09:12:40.170967+0000 (oldest deadline 2024-03-21T09:13:03.671007+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:06.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:06.233 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:06.233 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:05 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:05.988+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:06.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:06.144+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:06.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:06.144+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:06.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:06.144+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:06.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:06 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:06.144+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:07.172 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:06 smithi105 podman[73242]: 2024-03-21 09:13:06.858319637 +0000 UTC m=+5.528103225 container died 4df4d0b2452942e1278c87558c0014e019067a6a156d31fbfa0fd32189d1f4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:07.172 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:07 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:07.004+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:07.172 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:07 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:07.004+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:07.173 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:07 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:07.004+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:07.173 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:07 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:07.004+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:07.173 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:07 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:07.004+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:07.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:06 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:06.776+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:07.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:06 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:06.776+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:07.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:06 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:06.776+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:07.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:06 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:06.777+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:07.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:06 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:06.777+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:07.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:07 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:07.109+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:07.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:07 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:07.109+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:07.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:07 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:07.109+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:07.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:07 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:07.109+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:07.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:07 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:07.109+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:07.481 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:07 smithi105 podman[73242]: 2024-03-21 09:13:07.179435888 +0000 UTC m=+5.849219470 container cleanup 4df4d0b2452942e1278c87558c0014e019067a6a156d31fbfa0fd32189d1f4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-21T09:13:07.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:07 smithi105 bash[73242]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8 2024-03-21T09:13:07.482 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:07 smithi105 podman[73260]: 2024-03-21 09:13:07.418210392 +0000 UTC m=+0.559930943 container remove 4df4d0b2452942e1278c87558c0014e019067a6a156d31fbfa0fd32189d1f4cd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-03-21T09:13:08.135 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:07 smithi105 podman[73348]: 2024-03-21 09:13:07.741090798 +0000 UTC m=+0.017530910 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:13:08.135 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:07 smithi105 podman[73348]: 2024-03-21 09:13:07.856634693 +0000 UTC m=+0.133074813 container create 93737c98944f0729734e46bf8b0977f8822bb76028dff28085f9e7fd6ac646b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:08.136 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.003+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:08.136 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.003+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:08.136 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.003+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:08.136 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.003+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:08.136 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.003+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:08.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:07 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:07.823+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:08.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:07 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:07.823+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:08.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:07 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:07.823+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:08.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:07 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:07.823+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:08.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:07 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:07.823+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:08.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:07 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:07.823+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:08.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:08.134+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:08.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:08.134+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:08.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:08.134+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:08.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:08.134+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:08.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:08 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:08.134+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:08.442 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:08 smithi105 podman[73348]: 2024-03-21 09:13:08.142437442 +0000 UTC m=+0.418877557 container init 93737c98944f0729734e46bf8b0977f8822bb76028dff28085f9e7fd6ac646b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:13:08.442 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:08 smithi105 podman[73348]: 2024-03-21 09:13:08.146280055 +0000 UTC m=+0.422720149 container start 93737c98944f0729734e46bf8b0977f8822bb76028dff28085f9e7fd6ac646b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-03-21T09:13:08.442 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:08 smithi105 podman[73348]: 2024-03-21 09:13:08.234214237 +0000 UTC m=+0.510654337 container attach 93737c98944f0729734e46bf8b0977f8822bb76028dff28085f9e7fd6ac646b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8-deactivate, org.label-schema.build-date=20240319, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2) 2024-03-21T09:13:08.731 INFO:journalctl@ceph.osd.8.smithi105.stdout:Mar 21 09:13:08 smithi105 podman[73348]: 2024-03-21 09:13:08.449449082 +0000 UTC m=+0.725889171 container died 93737c98944f0729734e46bf8b0977f8822bb76028dff28085f9e7fd6ac646b3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-8-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:09.014 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.979+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:09.014 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.979+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:09.014 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.979+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:09.014 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.979+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:09.014 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:08.979+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:09.043 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.8.service' 2024-03-21T09:13:09.088 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:09.088 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-03-21T09:13:09.088 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-03-21T09:13:09.088 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.9 2024-03-21T09:13:09.266 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:08 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:08.828+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:09.266 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:08 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:08.828+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:09.266 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:08 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:08.828+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:09.267 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:08 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:08.828+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:09.267 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:08 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:08.828+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:09.267 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:08 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:08.828+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:09.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 systemd[1]: Stopping Ceph osd.9 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:09.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.158+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:09.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.158+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:09.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.158+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:09.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.158+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:09.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.158+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:09.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.428+0000 7fcb9de3e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:09.905 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.428+0000 7fcb9de3e700 -1 osd.9 91 *** Got signal Terminated *** 2024-03-21T09:13:09.905 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:09 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:09.428+0000 7fcb9de3e700 -1 osd.9 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:13:10.223 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:09 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:09.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:10.223 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:09 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:09.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:10.223 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:09 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:09.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:10.223 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:09 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:09.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:10.223 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:09 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:09.938+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:10.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:09 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:09.807+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:10.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:09 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:09.807+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:10.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:09 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:09.807+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:10.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:09 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:09.807+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:10.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:09 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:09.807+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:10.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:09 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:09.807+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:10.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:10 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:10.111+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:10.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:10 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:10.111+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:10.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:10 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:10.111+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:10.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:10 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:10.111+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:10.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:10 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:10.111+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:11.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:10 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:10.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:11.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:10 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:10.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:11.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:10 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:10.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:11.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:10 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:10.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:11.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:10 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:10.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:11.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:10 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:10.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:11.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:10.986+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:11.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:10.986+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:11.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:10.986+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:11.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:10.986+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:11.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:10 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:10.986+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:11.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:11.064+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:11.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:11.064+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:11.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:11.064+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:11.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:11.064+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:11.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:11 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:11.064+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:12.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:11.955+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:12.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:11.955+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:12.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:11.955+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:12.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:11.955+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:12.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:11.955+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:12.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:11.955+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:12.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:11 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:11.784+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:12.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:11 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:11.784+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:12.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:11 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:11.784+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:12.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:11 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:11.784+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:12.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:11 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:11.784+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:12.266 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:11 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:11.784+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:12.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:12.087+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:12.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:12.087+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:12.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:12.087+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:12.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:12.087+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:12.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:12 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:12.087+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:13.231 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:12.956+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:13.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:12.956+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:13.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:12.956+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:13.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:12.956+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:13.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:12.956+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:13.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:12 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:12.956+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:13.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:12 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:12.821+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:13.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:12 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:12.821+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:13.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:12 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:12.821+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:13.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:12 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:12.821+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:13.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:12 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:12.821+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:13.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:12 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:12.821+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:13.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:13.116+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:13.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:13.116+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:13.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:13.116+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:13.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:13.116+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:13.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:13.116+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:13.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:13 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:13.116+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:48.469142+0000 front 2024-03-21T09:12:48.469162+0000 (oldest deadline 2024-03-21T09:13:12.569195+0000) 2024-03-21T09:13:14.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:13.949+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:14.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:13.949+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:14.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:13.949+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:14.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:13.949+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:14.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:13.949+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:14.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:13 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:13.949+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:14.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:13.788+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:14.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:13.788+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:14.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:13.788+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:14.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:13.788+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:14.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:13.788+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:14.265 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:13 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:13.788+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:14.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:14.066+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:58.460065+0000 front 2024-03-21T09:11:58.460032+0000 (oldest deadline 2024-03-21T09:12:22.560233+0000) 2024-03-21T09:13:14.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:14.066+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.261509+0000 front 2024-03-21T09:12:07.261614+0000 (oldest deadline 2024-03-21T09:12:31.262168+0000) 2024-03-21T09:13:14.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:14.066+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:17.564341+0000 front 2024-03-21T09:12:17.564490+0000 (oldest deadline 2024-03-21T09:12:41.664150+0000) 2024-03-21T09:13:14.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:14.066+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:26.365183+0000 front 2024-03-21T09:12:26.365225+0000 (oldest deadline 2024-03-21T09:12:47.465383+0000) 2024-03-21T09:13:14.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:14.066+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:39.767936+0000 front 2024-03-21T09:12:39.767946+0000 (oldest deadline 2024-03-21T09:13:06.168216+0000) 2024-03-21T09:13:14.405 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9[67323]: 2024-03-21T09:13:14.066+0000 7fcb9642a700 -1 osd.9 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:48.469142+0000 front 2024-03-21T09:12:48.469162+0000 (oldest deadline 2024-03-21T09:13:12.569195+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:15.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:14 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:14.743+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:15.154 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:14 smithi087 podman[89636]: 2024-03-21 09:13:14.898467366 +0000 UTC m=+5.605167893 container died f373877be474e07e32feb1dd47af2d422f0b0bf1ad9dbec7b9bb559f811f35ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:13:15.231 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:14.925+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:15.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:14.925+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:15.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:14.925+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:15.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:14.925+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:15.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:14.925+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:15.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:14 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:14.925+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:15.541 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:15 smithi087 podman[89636]: 2024-03-21 09:13:15.264210645 +0000 UTC m=+5.970911170 container cleanup f373877be474e07e32feb1dd47af2d422f0b0bf1ad9dbec7b9bb559f811f35ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-03-21T09:13:15.542 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:15 smithi087 bash[89636]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9 2024-03-21T09:13:15.831 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:15 smithi087 podman[89653]: 2024-03-21 09:13:15.548734408 +0000 UTC m=+0.642033091 container remove f373877be474e07e32feb1dd47af2d422f0b0bf1ad9dbec7b9bb559f811f35ae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-03-21T09:13:16.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:16.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:16.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:16.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:16.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:16.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:16.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:15 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:15.719+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:16.136 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:15 smithi087 podman[89739]: 2024-03-21 09:13:15.74182071 +0000 UTC m=+0.017616186 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:13:16.137 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:15 smithi087 podman[89739]: 2024-03-21 09:13:15.849463585 +0000 UTC m=+0.125259067 container create 8d88b3cab2b64949789259b51228655d8aa99248bf7192d7bf4b947170c52ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-03-21T09:13:16.137 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 podman[89739]: 2024-03-21 09:13:16.06866432 +0000 UTC m=+0.344459805 container init 8d88b3cab2b64949789259b51228655d8aa99248bf7192d7bf4b947170c52ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319) 2024-03-21T09:13:16.137 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 podman[89739]: 2024-03-21 09:13:16.071898795 +0000 UTC m=+0.347694280 container start 8d88b3cab2b64949789259b51228655d8aa99248bf7192d7bf4b947170c52ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:16.137 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 podman[89739]: 2024-03-21 09:13:16.143615258 +0000 UTC m=+0.419410737 container attach 8d88b3cab2b64949789259b51228655d8aa99248bf7192d7bf4b947170c52ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5) 2024-03-21T09:13:16.231 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:15.935+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:16.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:15.935+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:16.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:15.935+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:16.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:15.935+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:16.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:15.935+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:16.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:15.935+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:16.404 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 podman[89739]: 2024-03-21 09:13:16.359736053 +0000 UTC m=+0.635531530 container died 8d88b3cab2b64949789259b51228655d8aa99248bf7192d7bf4b947170c52ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:13:16.887 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.9.service' 2024-03-21T09:13:16.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 podman[89765]: 2024-03-21 09:13:16.872519476 +0000 UTC m=+0.502808108 container remove 8d88b3cab2b64949789259b51228655d8aa99248bf7192d7bf4b947170c52ffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-9-deactivate, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-03-21T09:13:16.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.9.service: Deactivated successfully. 2024-03-21T09:13:16.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 systemd[1]: Stopped Ceph osd.9 for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T09:13:16.904 INFO:journalctl@ceph.osd.9.smithi087.stdout:Mar 21 09:13:16 smithi087 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.9.service: Consumed 7min 35.617s CPU time. 2024-03-21T09:13:16.929 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:16.930 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-03-21T09:13:16.930 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-03-21T09:13:16.930 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.10 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:16.961 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:16.715+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:17.231 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:16.941+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:17.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:16.941+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:17.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:16.941+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:17.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:16.941+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:17.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:16.941+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:17.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:16 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:16.941+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:17.251 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:16 smithi100 systemd[1]: Stopping Ceph osd.10 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:17.515 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.256+0000 7fc0e057b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:17.515 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.256+0000 7fc0e057b700 -1 osd.10 91 *** Got signal Terminated *** 2024-03-21T09:13:17.515 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.256+0000 7fc0e057b700 -1 osd.10 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:13:18.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:18.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:18.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:18.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:18.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:18.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:18.016 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:17 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:17.709+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:18.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:17 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:17.920+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:18.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:17 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:17.920+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:18.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:17 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:17.920+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:18.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:17 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:17.920+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:18.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:17 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:17.920+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:18.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:17 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:17.920+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:19.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:18 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:18.759+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:19.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:18 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:18.932+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:19.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:18 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:18.932+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:19.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:18 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:18.932+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:19.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:18 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:18.932+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:19.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:18 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:18.932+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:19.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:18 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:18.932+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:19.982 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:19 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:19.947+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:20.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:19 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:19.766+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:20.998 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:20.999 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:20.999 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:20.999 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:20.999 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:20.999 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:20.999 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:20 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:20.981+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:21.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:20 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:20.762+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:12:00.164553+0000 front 2024-03-21T09:12:00.164484+0000 (oldest deadline 2024-03-21T09:12:25.965136+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:05.965415+0000 front 2024-03-21T09:12:05.965499+0000 (oldest deadline 2024-03-21T09:12:31.865526+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:14.166442+0000 front 2024-03-21T09:12:14.166704+0000 (oldest deadline 2024-03-21T09:12:38.866452+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6814 osd.3 since back 2024-03-21T09:12:23.968156+0000 front 2024-03-21T09:12:23.968085+0000 (oldest deadline 2024-03-21T09:12:48.668107+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:33.969039+0000 front 2024-03-21T09:12:33.969111+0000 (oldest deadline 2024-03-21T09:12:59.269209+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:42.070922+0000 front 2024-03-21T09:12:42.070911+0000 (oldest deadline 2024-03-21T09:13:07.370705+0000) 2024-03-21T09:13:22.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:21 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10[60943]: 2024-03-21T09:13:21.738+0000 7fc0dcb6f700 -1 osd.10 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:49.071683+0000 front 2024-03-21T09:12:49.071583+0000 (oldest deadline 2024-03-21T09:13:14.371454+0000) 2024-03-21T09:13:22.231 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:22.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:22.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:22.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:22.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:22.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:22.232 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:22 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:21.999+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:23.015 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:22 smithi100 podman[71855]: 2024-03-21 09:13:22.693527461 +0000 UTC m=+5.564885746 container died 2d7846dffb60b09a38a19438be88d2b438cf941b0b493d86551b038108d6f064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-03-21T09:13:23.417 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:23 smithi100 podman[71855]: 2024-03-21 09:13:23.214548639 +0000 UTC m=+6.085906943 container cleanup 2d7846dffb60b09a38a19438be88d2b438cf941b0b493d86551b038108d6f064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240319, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:13:23.417 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:23 smithi100 bash[71855]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10 2024-03-21T09:13:23.481 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:23.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:23.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:23.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:23.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:23.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:23.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:23 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:23.030+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:23.765 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:23 smithi100 podman[71872]: 2024-03-21 09:13:23.423891288 +0000 UTC m=+0.726435838 container remove 2d7846dffb60b09a38a19438be88d2b438cf941b0b493d86551b038108d6f064 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:13:24.112 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:23 smithi100 podman[71963]: 2024-03-21 09:13:23.721842716 +0000 UTC m=+0.017844939 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:13:24.112 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:23 smithi100 podman[71963]: 2024-03-21 09:13:23.833188337 +0000 UTC m=+0.129190544 container create 46d1d45de8050f619ade826363234f37d9dc4985261aa1c2963203c75d075b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319) 2024-03-21T09:13:24.112 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:24 smithi100 podman[71963]: 2024-03-21 09:13:24.035712491 +0000 UTC m=+0.331714710 container init 46d1d45de8050f619ade826363234f37d9dc4985261aa1c2963203c75d075b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6) 2024-03-21T09:13:24.112 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:24 smithi100 podman[71963]: 2024-03-21 09:13:24.039279255 +0000 UTC m=+0.335281456 container start 46d1d45de8050f619ade826363234f37d9dc4985261aa1c2963203c75d075b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:13:24.112 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:24 smithi100 podman[71963]: 2024-03-21 09:13:24.118856893 +0000 UTC m=+0.414859111 container attach 46d1d45de8050f619ade826363234f37d9dc4985261aa1c2963203c75d075b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:13:24.481 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:24.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:24.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:24.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:24.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:24.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:24.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:24 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:24.076+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:24.515 INFO:journalctl@ceph.osd.10.smithi100.stdout:Mar 21 09:13:24 smithi100 podman[71963]: 2024-03-21 09:13:24.327813926 +0000 UTC m=+0.623816129 container died 46d1d45de8050f619ade826363234f37d9dc4985261aa1c2963203c75d075b7c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-03-21T09:13:24.883 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.10.service' 2024-03-21T09:13:24.964 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:24.964 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-03-21T09:13:24.965 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-03-21T09:13:24.965 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.11 2024-03-21T09:13:25.267 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 systemd[1]: Stopping Ceph osd.11 for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.088+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.273+0000 7f18235b4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.273+0000 7f18235b4700 -1 osd.11 91 *** Got signal Terminated *** 2024-03-21T09:13:25.268 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:25 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:25.273+0000 7f18235b4700 -1 osd.11 91 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:26.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:26 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:26.104+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:27.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:27 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:27.152+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:28.481 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:28.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:28.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:28.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:28.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:28.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:28.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:28 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:28.183+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:29.481 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:29.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:29.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:29.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:29.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:29.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:29.482 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:29 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:29.211+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6806 osd.0 since back 2024-03-21T09:11:59.283966+0000 front 2024-03-21T09:11:59.283763+0000 (oldest deadline 2024-03-21T09:12:25.183732+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6804 osd.1 since back 2024-03-21T09:12:07.484657+0000 front 2024-03-21T09:12:07.484639+0000 (oldest deadline 2024-03-21T09:12:32.184721+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6804 osd.2 since back 2024-03-21T09:12:18.085697+0000 front 2024-03-21T09:12:18.085717+0000 (oldest deadline 2024-03-21T09:12:40.385858+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6812 osd.4 since back 2024-03-21T09:12:30.689423+0000 front 2024-03-21T09:12:30.689351+0000 (oldest deadline 2024-03-21T09:12:55.989326+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6812 osd.5 since back 2024-03-21T09:12:40.590699+0000 front 2024-03-21T09:12:40.590762+0000 (oldest deadline 2024-03-21T09:13:03.490906+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.87:6822 osd.6 since back 2024-03-21T09:12:46.992137+0000 front 2024-03-21T09:12:46.992152+0000 (oldest deadline 2024-03-21T09:13:11.091941+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.100:6820 osd.7 since back 2024-03-21T09:12:57.894444+0000 front 2024-03-21T09:12:57.894478+0000 (oldest deadline 2024-03-21T09:13:19.594330+0000) 2024-03-21T09:13:30.732 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11[61632]: 2024-03-21T09:13:30.244+0000 7f181fba8700 -1 osd.11 91 heartbeat_check: no reply from 172.21.15.105:6820 osd.8 since back 2024-03-21T09:13:04.195630+0000 front 2024-03-21T09:13:04.195661+0000 (oldest deadline 2024-03-21T09:13:30.095662+0000) 2024-03-21T09:13:31.025 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:30 smithi105 podman[73476]: 2024-03-21 09:13:30.7692508 +0000 UTC m=+5.600418814 container died 5cd67200ce2fc2377042a95d54d3c858742f6fe1d4b72e2dbe343401e793d326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-03-21T09:13:31.295 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:31 smithi105 podman[73476]: 2024-03-21 09:13:31.117495659 +0000 UTC m=+5.948663676 container cleanup 5cd67200ce2fc2377042a95d54d3c858742f6fe1d4b72e2dbe343401e793d326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux ) 2024-03-21T09:13:31.295 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:31 smithi105 bash[73476]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11 2024-03-21T09:13:31.702 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:31 smithi105 podman[73493]: 2024-03-21 09:13:31.301935828 +0000 UTC m=+0.528563534 container remove 5cd67200ce2fc2377042a95d54d3c858742f6fe1d4b72e2dbe343401e793d326 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240319, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS) 2024-03-21T09:13:31.985 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:31 smithi105 podman[73582]: 2024-03-21 09:13:31.612524759 +0000 UTC m=+0.017581732 image pull 237e43c7a65ff48534d94b97c1bfdbe4b7be5672e3d9a613745083ea3702aa54 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232 2024-03-21T09:13:31.986 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:31 smithi105 podman[73582]: 2024-03-21 09:13:31.794732221 +0000 UTC m=+0.199789181 container create 5d6196ee99aa88eb989913ec0a752ba31d812ede9bcd56f9b4bdb9320146ee7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11-deactivate, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:32.343 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:32 smithi105 podman[73582]: 2024-03-21 09:13:32.064083593 +0000 UTC m=+0.469140554 container init 5d6196ee99aa88eb989913ec0a752ba31d812ede9bcd56f9b4bdb9320146ee7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11-deactivate, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-03-21T09:13:32.343 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:32 smithi105 podman[73582]: 2024-03-21 09:13:32.067298318 +0000 UTC m=+0.472355288 container start 5d6196ee99aa88eb989913ec0a752ba31d812ede9bcd56f9b4bdb9320146ee7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-03-21T09:13:32.343 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:32 smithi105 podman[73582]: 2024-03-21 09:13:32.147223601 +0000 UTC m=+0.552280575 container attach 5d6196ee99aa88eb989913ec0a752ba31d812ede9bcd56f9b4bdb9320146ee7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11-deactivate, org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-03-21T09:13:32.635 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:32 smithi105 podman[73582]: 2024-03-21 09:13:32.350658398 +0000 UTC m=+0.755715350 container died 5d6196ee99aa88eb989913ec0a752ba31d812ede9bcd56f9b4bdb9320146ee7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:32.891 INFO:journalctl@ceph.osd.11.smithi105.stdout:Mar 21 09:13:32 smithi105 podman[73609]: 2024-03-21 09:13:32.883772517 +0000 UTC m=+0.527832086 container remove 5d6196ee99aa88eb989913ec0a752ba31d812ede9bcd56f9b4bdb9320146ee7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240319, GIT_BRANCH=HEAD, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:13:32.897 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@osd.11.service' 2024-03-21T09:13:32.938 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:32.939 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-03-21T09:13:32.939 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-03-21T09:13:32.939 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.a 2024-03-21T09:13:33.242 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:13:32 smithi087 systemd[1]: Stopping Ceph mds.a for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:33.507 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:13:33 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a[73694]: 2024-03-21T09:13:33.248+0000 7f55de528700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:33.507 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:13:33 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a[73694]: 2024-03-21T09:13:33.248+0000 7f55de528700 -1 mds.a *** got signal Terminated *** 2024-03-21T09:13:35.085 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:13:34 smithi087 podman[89843]: 2024-03-21 09:13:34.817387958 +0000 UTC m=+1.677619486 container died d1d898c1ca914c20a983fe421eaeff9191f3fb1eeb73f3f87684ad0ccc617f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a, org.label-schema.vendor=CentOS, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:35.404 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:13:35 smithi087 podman[89843]: 2024-03-21 09:13:35.190836599 +0000 UTC m=+2.051068132 container cleanup d1d898c1ca914c20a983fe421eaeff9191f3fb1eeb73f3f87684ad0ccc617f2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:13:35.404 INFO:journalctl@ceph.mds.a.smithi087.stdout:Mar 21 09:13:35 smithi087 bash[89843]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-a 2024-03-21T09:13:35.556 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.a.service' 2024-03-21T09:13:35.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:35.594 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-03-21T09:13:35.594 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-03-21T09:13:35.594 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.d 2024-03-21T09:13:35.904 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 09:13:35 smithi087 systemd[1]: Stopping Ceph mds.d for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:36.404 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 09:13:35 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d[72743]: 2024-03-21T09:13:35.936+0000 7f8199e16700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:36.404 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 09:13:35 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d[72743]: 2024-03-21T09:13:35.936+0000 7f8199e16700 -1 mds.d *** got signal Terminated *** 2024-03-21T09:13:38.397 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 09:13:38 smithi087 podman[89974]: 2024-03-21 09:13:38.087191891 +0000 UTC m=+2.258525247 container died 11bdb4052c12681f226536db9a9f321d4a304921977d704c99c882c4f01ed492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-03-21T09:13:38.654 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 09:13:38 smithi087 podman[89974]: 2024-03-21 09:13:38.486502451 +0000 UTC m=+2.657835810 container cleanup 11bdb4052c12681f226536db9a9f321d4a304921977d704c99c882c4f01ed492 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:13:38.654 INFO:journalctl@ceph.mds.d.smithi087.stdout:Mar 21 09:13:38 smithi087 bash[89974]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-d 2024-03-21T09:13:38.847 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.d.service' 2024-03-21T09:13:38.883 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:38.884 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-03-21T09:13:38.884 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-03-21T09:13:38.884 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.g 2024-03-21T09:13:39.154 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:13:38 smithi087 systemd[1]: Stopping Ceph mds.g for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:39.654 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:13:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g[73351]: 2024-03-21T09:13:39.248+0000 7f9cd6b54700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:39.654 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:13:39 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g[73351]: 2024-03-21T09:13:39.248+0000 7f9cd6b54700 -1 mds.g *** got signal Terminated *** 2024-03-21T09:13:43.051 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:13:42 smithi087 podman[90097]: 2024-03-21 09:13:42.793616364 +0000 UTC m=+3.680666338 container died d27be41e4d22d070d64085d990f4290fb31a824bf3f5bfe2057ac2265887b466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-03-21T09:13:43.404 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:13:43 smithi087 podman[90097]: 2024-03-21 09:13:43.148568166 +0000 UTC m=+4.035618193 container cleanup d27be41e4d22d070d64085d990f4290fb31a824bf3f5bfe2057ac2265887b466 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-03-21T09:13:43.404 INFO:journalctl@ceph.mds.g.smithi087.stdout:Mar 21 09:13:43 smithi087 bash[90097]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-g 2024-03-21T09:13:43.558 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.g.service' 2024-03-21T09:13:43.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:43.593 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-03-21T09:13:43.593 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-03-21T09:13:43.593 DEBUG:teuthology.orchestra.run.smithi087:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.j 2024-03-21T09:13:43.904 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:43 smithi087 systemd[1]: Stopping Ceph mds.j for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:44.154 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-j[72397]: 2024-03-21T09:13:43.910+0000 7fe43540c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:44.155 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:43 smithi087 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-j[72397]: 2024-03-21T09:13:43.910+0000 7fe43540c700 -1 mds.j *** got signal Terminated *** 2024-03-21T09:13:47.436 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:47 smithi087 podman[90218]: 2024-03-21 09:13:47.19012851 +0000 UTC m=+3.376320010 container died b7c8e830bb6748fa195ebaed22a11fb7dc84b937aea046c1d6907d93adf0be0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-j, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:47.885 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:47 smithi087 podman[90218]: 2024-03-21 09:13:47.527646012 +0000 UTC m=+3.713837510 container cleanup b7c8e830bb6748fa195ebaed22a11fb7dc84b937aea046c1d6907d93adf0be0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-j, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240319) 2024-03-21T09:13:47.885 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:47 smithi087 bash[90218]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-j 2024-03-21T09:13:47.885 INFO:journalctl@ceph.mds.j.smithi087.stdout:Mar 21 09:13:47 smithi087 podman[90233]: 2024-03-21 09:13:47.686994109 +0000 UTC m=+0.492017268 container remove b7c8e830bb6748fa195ebaed22a11fb7dc84b937aea046c1d6907d93adf0be0b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-j, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:47.896 DEBUG:teuthology.orchestra.run.smithi087:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.j.service' 2024-03-21T09:13:47.931 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:47.931 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-03-21T09:13:47.931 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-03-21T09:13:47.931 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.b 2024-03-21T09:13:48.217 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:13:47 smithi100 systemd[1]: Stopping Ceph mds.b for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:48.514 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:13:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b[66199]: 2024-03-21T09:13:48.222+0000 7f4778227700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:48.515 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:13:48 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b[66199]: 2024-03-21T09:13:48.222+0000 7f4778227700 -1 mds.b *** got signal Terminated *** 2024-03-21T09:13:51.015 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:13:50 smithi100 podman[72080]: 2024-03-21 09:13:50.766022318 +0000 UTC m=+2.639655734 container died 32d8de4201f7ab16311c3d02a6524ad95712da9a60fa7bbb65e58d4ddae965f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:13:51.425 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:13:51 smithi100 podman[72080]: 2024-03-21 09:13:51.131645379 +0000 UTC m=+3.005278769 container cleanup 32d8de4201f7ab16311c3d02a6524ad95712da9a60fa7bbb65e58d4ddae965f4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:13:51.425 INFO:journalctl@ceph.mds.b.smithi100.stdout:Mar 21 09:13:51 smithi100 bash[72080]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-b 2024-03-21T09:13:51.588 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.b.service' 2024-03-21T09:13:51.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:51.624 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-03-21T09:13:51.625 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-03-21T09:13:51.625 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.e 2024-03-21T09:13:51.962 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:13:51 smithi100 systemd[1]: Stopping Ceph mds.e for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:52.264 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:13:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e[65573]: 2024-03-21T09:13:51.967+0000 7f5e85dd4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:52.265 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:13:51 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e[65573]: 2024-03-21T09:13:51.967+0000 7f5e85dd4700 -1 mds.e *** got signal Terminated *** 2024-03-21T09:13:55.038 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:13:54 smithi100 podman[72200]: 2024-03-21 09:13:54.77691624 +0000 UTC m=+2.918191445 container died ae2d741befc59e7d84e3229b7292bfb5888e6c65327f76b1efabace934179ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, org.label-schema.build-date=20240319) 2024-03-21T09:13:55.421 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:13:55 smithi100 podman[72200]: 2024-03-21 09:13:55.143693473 +0000 UTC m=+3.284968666 container cleanup ae2d741befc59e7d84e3229b7292bfb5888e6c65327f76b1efabace934179ac0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:13:55.421 INFO:journalctl@ceph.mds.e.smithi100.stdout:Mar 21 09:13:55 smithi100 bash[72200]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-e 2024-03-21T09:13:55.559 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.e.service' 2024-03-21T09:13:55.594 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:55.594 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-03-21T09:13:55.594 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-03-21T09:13:55.594 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.h 2024-03-21T09:13:56.015 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:13:55 smithi100 systemd[1]: Stopping Ceph mds.h for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:13:56.015 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:13:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h[65893]: 2024-03-21T09:13:55.905+0000 7fe0eb1c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:13:56.015 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:13:55 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h[65893]: 2024-03-21T09:13:55.905+0000 7fe0eb1c0700 -1 mds.h *** got signal Terminated *** 2024-03-21T09:13:59.015 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:13:58 smithi100 podman[72320]: 2024-03-21 09:13:58.766132566 +0000 UTC m=+2.952498550 container died fff9dbdc6e87c9bc16ede6f61fd5252812d7e46a71b3bd4b0d55d7c06a42c4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-03-21T09:13:59.382 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:13:59 smithi100 podman[72320]: 2024-03-21 09:13:59.113746414 +0000 UTC m=+3.300112385 container cleanup fff9dbdc6e87c9bc16ede6f61fd5252812d7e46a71b3bd4b0d55d7c06a42c4e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.build-date=20240319, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:13:59.382 INFO:journalctl@ceph.mds.h.smithi100.stdout:Mar 21 09:13:59 smithi100 bash[72320]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-h 2024-03-21T09:13:59.527 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.h.service' 2024-03-21T09:13:59.561 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:13:59.562 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-03-21T09:13:59.562 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-03-21T09:13:59.562 DEBUG:teuthology.orchestra.run.smithi100:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.k 2024-03-21T09:14:00.014 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:13:59 smithi100 systemd[1]: Stopping Ceph mds.k for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:14:00.514 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:14:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k[65254]: 2024-03-21T09:14:00.034+0000 7fc36d018700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:14:00.515 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:14:00 smithi100 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k[65254]: 2024-03-21T09:14:00.034+0000 7fc36d018700 -1 mds.k *** got signal Terminated *** 2024-03-21T09:14:03.114 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:14:02 smithi100 podman[72441]: 2024-03-21 09:14:02.842607743 +0000 UTC m=+3.063036813 container died 69d3153453cac6a2c8a0eb7fa5cc380f01d21d9a535f1b204682f7384b3f7e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240319) 2024-03-21T09:14:03.453 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:14:03 smithi100 podman[72441]: 2024-03-21 09:14:03.209199018 +0000 UTC m=+3.429628068 container cleanup 69d3153453cac6a2c8a0eb7fa5cc380f01d21d9a535f1b204682f7384b3f7e03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240319, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-03-21T09:14:03.453 INFO:journalctl@ceph.mds.k.smithi100.stdout:Mar 21 09:14:03 smithi100 bash[72441]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-k 2024-03-21T09:14:03.621 DEBUG:teuthology.orchestra.run.smithi100:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.k.service' 2024-03-21T09:14:03.655 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:14:03.656 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-03-21T09:14:03.656 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-03-21T09:14:03.656 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.c 2024-03-21T09:14:03.969 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:03 smithi105 systemd[1]: Stopping Ceph mds.c for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:14:04.231 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c[66941]: 2024-03-21T09:14:03.976+0000 7fa36dbd7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:14:04.231 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:03 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c[66941]: 2024-03-21T09:14:03.976+0000 7fa36dbd7700 -1 mds.c *** got signal Terminated *** 2024-03-21T09:14:07.153 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:06 smithi105 podman[73689]: 2024-03-21 09:14:06.827452604 +0000 UTC m=+2.976760850 container died bc8bc523e9fb99869075139b2eacadb14a0b84b2e1489b07edd728343bbb52ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-03-21T09:14:07.469 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:07 smithi105 podman[73689]: 2024-03-21 09:14:07.259017595 +0000 UTC m=+3.408325848 container cleanup bc8bc523e9fb99869075139b2eacadb14a0b84b2e1489b07edd728343bbb52ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-03-21T09:14:07.470 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:07 smithi105 bash[73689]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c 2024-03-21T09:14:07.727 INFO:journalctl@ceph.mds.c.smithi105.stdout:Mar 21 09:14:07 smithi105 podman[73716]: 2024-03-21 09:14:07.4768509 +0000 UTC m=+0.650582234 container remove bc8bc523e9fb99869075139b2eacadb14a0b84b2e1489b07edd728343bbb52ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-c, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, ceph=True, org.label-schema.build-date=20240319, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:14:07.731 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.c.service' 2024-03-21T09:14:07.769 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:14:07.769 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-03-21T09:14:07.769 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-03-21T09:14:07.769 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.f 2024-03-21T09:14:07.982 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:07 smithi105 systemd[1]: Stopping Ceph mds.f for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:14:08.481 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f[66632]: 2024-03-21T09:14:08.078+0000 7fcbc45a5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:14:08.481 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:08 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f[66632]: 2024-03-21T09:14:08.078+0000 7fcbc45a5700 -1 mds.f *** got signal Terminated *** 2024-03-21T09:14:11.082 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:10 smithi105 podman[73824]: 2024-03-21 09:14:10.826615387 +0000 UTC m=+2.836801054 container died 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f, org.label-schema.schema-version=1.0, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-03-21T09:14:11.340 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:11 smithi105 podman[73824]: 2024-03-21 09:14:11.170952253 +0000 UTC m=+3.181137911 container cleanup 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-03-21T09:14:11.340 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:11 smithi105 bash[73824]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f 2024-03-21T09:14:11.577 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.f.service' 2024-03-21T09:14:11.601 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:11 smithi105 podman[73838]: 2024-03-21 09:14:11.347169309 +0000 UTC m=+0.515954380 container remove 239b5b7b71bb33debae8dbb2dd2d142416f224fd701f045a93aa4ad3fd0fbc70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-f, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240319, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.schema-version=1.0, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:14:11.601 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:11 smithi105 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.f.service: Deactivated successfully. 2024-03-21T09:14:11.601 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:11 smithi105 systemd[1]: Stopped Ceph mds.f for 75a2f1f0-e75f-11ee-95cd-87774f69a715. 2024-03-21T09:14:11.601 INFO:journalctl@ceph.mds.f.smithi105.stdout:Mar 21 09:14:11 smithi105 systemd[1]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.f.service: Consumed 5min 15.278s CPU time. 2024-03-21T09:14:11.613 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:14:11.613 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-03-21T09:14:11.613 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-03-21T09:14:11.613 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.i 2024-03-21T09:14:11.934 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:14:11 smithi105 systemd[1]: Stopping Ceph mds.i for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:14:12.231 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:14:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: 2024-03-21T09:14:11.940+0000 7f7efae2d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:14:12.231 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:14:11 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i[67245]: 2024-03-21T09:14:11.940+0000 7f7efae2d700 -1 mds.i *** got signal Terminated *** 2024-03-21T09:14:15.101 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:14:14 smithi105 podman[73946]: 2024-03-21 09:14:14.824801083 +0000 UTC m=+2.993635159 container died 5c4370e9bdfeebd964f3c313ffc174573f48eda404ca9639a9ab6c7102ab8ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.build-date=20240319, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-03-21T09:14:15.460 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:14:15 smithi105 podman[73946]: 2024-03-21 09:14:15.208194384 +0000 UTC m=+3.377028466 container cleanup 5c4370e9bdfeebd964f3c313ffc174573f48eda404ca9639a9ab6c7102ab8ad4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5) 2024-03-21T09:14:15.460 INFO:journalctl@ceph.mds.i.smithi105.stdout:Mar 21 09:14:15 smithi105 bash[73946]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-i 2024-03-21T09:14:15.559 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.i.service' 2024-03-21T09:14:15.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:14:15.593 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-03-21T09:14:15.594 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-03-21T09:14:15.594 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.l 2024-03-21T09:14:15.731 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:15 smithi105 systemd[1]: Stopping Ceph mds.l for 75a2f1f0-e75f-11ee-95cd-87774f69a715... 2024-03-21T09:14:16.231 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l[66305]: 2024-03-21T09:14:15.894+0000 7f098cebc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-03-21T09:14:16.231 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:15 smithi105 ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l[66305]: 2024-03-21T09:14:15.894+0000 7f098cebc700 -1 mds.l *** got signal Terminated *** 2024-03-21T09:14:19.130 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:18 smithi105 podman[74068]: 2024-03-21 09:14:18.827462192 +0000 UTC m=+3.011951990 container died fa46257e6b43384847a5a22884688eab830f77ad6930b8c9f8351e878deb7c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240319, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30) 2024-03-21T09:14:19.481 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:19 smithi105 podman[74068]: 2024-03-21 09:14:19.203583502 +0000 UTC m=+3.388073296 container cleanup fa46257e6b43384847a5a22884688eab830f77ad6930b8c9f8351e878deb7c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, io.buildah.version=1.33.5, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240319, org.label-schema.vendor=CentOS) 2024-03-21T09:14:19.481 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:19 smithi105 bash[74068]: ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l 2024-03-21T09:14:19.482 INFO:journalctl@ceph.mds.l.smithi105.stdout:Mar 21 09:14:19 smithi105 podman[74082]: 2024-03-21 09:14:19.379359155 +0000 UTC m=+0.521076496 container remove fa46257e6b43384847a5a22884688eab830f77ad6930b8c9f8351e878deb7c55 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:87c941b756786d43881503fb78c47c26d07136cca67af598d52ba7d6d9d06232, name=ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240319, GIT_COMMIT=08b1ab54e11946f162e232e2750e80887689df30, GIT_CLEAN=True, RELEASE=wip-vshankar-testing-20240320.053002-2edfee6, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-03-21T09:14:19.597 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-75a2f1f0-e75f-11ee-95cd-87774f69a715@mds.l.service' 2024-03-21T09:14:19.632 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-21T09:14:19.632 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-03-21T09:14:19.632 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T09:14:19.834 INFO:teuthology.orchestra.run.smithi087.stdout:Deleting cluster with fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T09:14:22.938 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T09:14:23.126 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T09:14:25.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --force --keep-logs 2024-03-21T09:14:25.939 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T09:14:28.418 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:14:28.453 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:14:28.488 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-21T09:14:28.517 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-21T09:14:28.548 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi087/crash 2024-03-21T09:14:28.549 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash -- . 2024-03-21T09:14:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:tar: /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T09:14:28.598 INFO:teuthology.orchestra.run.smithi087.stderr:tar: Error is not recoverable: exiting now 2024-03-21T09:14:28.600 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi100/crash 2024-03-21T09:14:28.601 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash -- . 2024-03-21T09:14:28.656 INFO:teuthology.orchestra.run.smithi100.stderr:tar: /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T09:14:28.657 INFO:teuthology.orchestra.run.smithi100.stderr:tar: Error is not recoverable: exiting now 2024-03-21T09:14:28.658 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi105/crash 2024-03-21T09:14:28.659 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash -- . 2024-03-21T09:14:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/crash: Cannot open: No such file or directory 2024-03-21T09:14:28.693 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-03-21T09:14:28.694 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-21T09:14:28.694 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-21T09:14:28.770 INFO:teuthology.orchestra.run.smithi087.stdout:2024-03-21T08:48:58.152621+0000 mon.a (mon.0) 188 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:14:28.771 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-21T09:14:28.771 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[SEC\]' /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-21T09:14:28.806 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[ERR\]' /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-21T09:14:28.880 DEBUG:teuthology.orchestra.run.smithi087:> sudo egrep '\[WRN\]' /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-03-21T09:14:28.953 INFO:teuthology.orchestra.run.smithi087.stdout:2024-03-21T08:48:58.152621+0000 mon.a (mon.0) 188 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c 2024-03-21T09:14:28.953 INFO:tasks.cephadm:Compressing logs... 2024-03-21T09:14:28.954 DEBUG:teuthology.orchestra.run.smithi087:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T09:14:28.996 DEBUG:teuthology.orchestra.run.smithi100:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T09:14:28.999 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-21T09:14:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T09:14:29.029 INFO:teuthology.orchestra.run.smithi087.stderr:: No such file or directory 2024-03-21T09:14:29.038 INFO:teuthology.orchestra.run.smithi100.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T09:14:29.039 INFO:teuthology.orchestra.run.smithi105.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-03-21T09:14:29.039 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T09:14:29.039 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.l.log 2024-03-21T09:14:29.039 INFO:teuthology.orchestra.run.smithi105.stderr:gzip/var/log/ceph/cephadm.log: -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.i.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.11.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.z.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.8.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.5.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.f.log 2024-03-21T09:14:29.043 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T09:14:29.044 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log 2024-03-21T09:14:29.044 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.2.log 2024-03-21T09:14:29.044 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.c.log 2024-03-21T09:14:29.044 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T09:14:29.044 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.c.log 2024-03-21T09:14:29.045 INFO:teuthology.orchestra.run.smithi087.stderr:gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.6.log 2024-03-21T09:14:29.045 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.a.log 2024-03-21T09:14:29.045 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T09:14:29.045 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.g.log 2024-03-21T09:14:29.046 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.c.log: 93.0% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.z.log.gz 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.j.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.3.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.x.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.0.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.9.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.d.log 2024-03-21T09:14:29.052 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.a.log 2024-03-21T09:14:29.053 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-21T09:14:29.053 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.10.log 2024-03-21T09:14:29.053 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log 2024-03-21T09:14:29.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/cephadm.log: /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.b.log 2024-03-21T09:14:29.053 INFO:teuthology.orchestra.run.smithi100.stderr:gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.4.log 2024-03-21T09:14:29.053 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log: /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.1.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.k.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.h.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.7.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.e.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.y.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log 2024-03-21T09:14:29.054 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.b.log 2024-03-21T09:14:29.055 INFO:teuthology.orchestra.run.smithi100.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log: /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.b.log: 93.0% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.y.log.gz 2024-03-21T09:14:29.084 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.f.log.gz 2024-03-21T09:14:29.088 INFO:teuthology.orchestra.run.smithi100.stderr: 93.9% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.e.log.gz 2024-03-21T09:14:29.112 INFO:teuthology.orchestra.run.smithi105.stderr: 93.8% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.l.log.gz 2024-03-21T09:14:29.121 INFO:teuthology.orchestra.run.smithi105.stderr: 82.6% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T09:14:29.127 INFO:teuthology.orchestra.run.smithi100.stderr: 82.6% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T09:14:29.129 INFO:teuthology.orchestra.run.smithi087.stderr:/var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.d.log: /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.a.log: 82.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.cephadm.log.gz 2024-03-21T09:14:29.145 INFO:teuthology.orchestra.run.smithi087.stderr: 94.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.d.log.gz 2024-03-21T09:14:29.146 INFO:teuthology.orchestra.run.smithi105.stderr: 90.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T09:14:29.164 INFO:teuthology.orchestra.run.smithi087.stderr: 90.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T09:14:29.164 INFO:teuthology.orchestra.run.smithi087.stderr: 87.0% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T09:14:29.170 INFO:teuthology.orchestra.run.smithi105.stderr: 87.1% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T09:14:29.184 INFO:teuthology.orchestra.run.smithi100.stderr: 90.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.audit.log.gz 2024-03-21T09:14:29.197 INFO:teuthology.orchestra.run.smithi105.stderr: 90.9% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T09:14:29.223 INFO:teuthology.orchestra.run.smithi100.stderr: 94.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.k.log.gz 2024-03-21T09:14:29.231 INFO:teuthology.orchestra.run.smithi100.stderr: 93.7% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.b.log.gz 2024-03-21T09:14:29.237 INFO:teuthology.orchestra.run.smithi105.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T09:14:29.244 INFO:teuthology.orchestra.run.smithi100.stderr: 94.7% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.h.log.gz 2024-03-21T09:14:29.292 INFO:teuthology.orchestra.run.smithi100.stderr: 91.0% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T09:14:29.333 INFO:teuthology.orchestra.run.smithi100.stderr: 87.1% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph.log.gz 2024-03-21T09:14:29.360 INFO:teuthology.orchestra.run.smithi087.stderr: 94.4% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.j.log.gz 2024-03-21T09:14:29.389 INFO:teuthology.orchestra.run.smithi087.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-21T09:14:29.392 INFO:teuthology.orchestra.run.smithi087.stderr: 91.0% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-volume.log.gz 2024-03-21T09:14:29.442 INFO:teuthology.orchestra.run.smithi087.stderr: 94.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.a.log.gz 2024-03-21T09:14:29.580 INFO:teuthology.orchestra.run.smithi087.stderr: 94.5% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.g.log.gz 2024-03-21T09:14:30.706 INFO:teuthology.orchestra.run.smithi087.stderr: 90.2% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mgr.x.log.gz 2024-03-21T09:14:30.816 INFO:teuthology.orchestra.run.smithi100.stderr: 92.1% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.b.log.gz 2024-03-21T09:14:34.179 INFO:teuthology.orchestra.run.smithi105.stderr: 91.9% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.c.log.gz 2024-03-21T09:14:35.073 INFO:teuthology.orchestra.run.smithi087.stderr: 90.2% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mon.a.log.gz 2024-03-21T09:17:48.396 INFO:teuthology.orchestra.run.smithi087.stderr: 93.2% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.6.log.gz 2024-03-21T09:19:45.060 INFO:teuthology.orchestra.run.smithi087.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.9.log.gz 2024-03-21T09:20:18.590 INFO:teuthology.orchestra.run.smithi087.stderr: 93.2% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.0.log.gz 2024-03-21T09:20:25.559 INFO:teuthology.orchestra.run.smithi087.stderr: 93.2% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.3.log.gz 2024-03-21T09:20:25.561 INFO:teuthology.orchestra.run.smithi087.stderr: 2024-03-21T09:20:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:real 5m56.550s 2024-03-21T09:20:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:user 4m51.452s 2024-03-21T09:20:25.562 INFO:teuthology.orchestra.run.smithi087.stderr:sys 0m13.077s 2024-03-21T09:22:12.320 INFO:teuthology.orchestra.run.smithi100.stderr: 93.2% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.7.log.gz 2024-03-21T09:23:06.448 INFO:teuthology.orchestra.run.smithi100.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.10.log.gz 2024-03-21T09:23:37.216 INFO:teuthology.orchestra.run.smithi100.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.1.log.gz 2024-03-21T09:23:45.015 INFO:teuthology.orchestra.run.smithi105.stderr: 97.4% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.c.log.gz 2024-03-21T09:23:54.834 INFO:teuthology.orchestra.run.smithi100.stderr: 93.4% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.4.log.gz 2024-03-21T09:23:54.893 INFO:teuthology.orchestra.run.smithi100.stderr: 2024-03-21T09:23:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:real 9m25.861s 2024-03-21T09:23:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:user 5m17.713s 2024-03-21T09:23:54.893 INFO:teuthology.orchestra.run.smithi100.stderr:sys 0m14.404s 2024-03-21T09:28:09.335 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.8.log.gz 2024-03-21T09:29:07.985 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.2.log.gz 2024-03-21T09:31:07.602 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.5.log.gz 2024-03-21T09:32:19.487 INFO:teuthology.orchestra.run.smithi105.stderr: 93.3% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-osd.11.log.gz 2024-03-21T09:34:56.882 INFO:teuthology.orchestra.run.smithi105.stderr: 94.4% -- replaced with /var/log/ceph/75a2f1f0-e75f-11ee-95cd-87774f69a715/ceph-mds.i.log.gz 2024-03-21T09:34:56.920 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-03-21T09:34:56.920 INFO:teuthology.orchestra.run.smithi105.stderr:real 20m27.900s 2024-03-21T09:34:56.920 INFO:teuthology.orchestra.run.smithi105.stderr:user 8m51.109s 2024-03-21T09:34:56.921 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m26.530s 2024-03-21T09:34:56.921 INFO:tasks.cephadm:Archiving logs... 2024-03-21T09:34:56.926 DEBUG:teuthology.misc:Transferring archived files from smithi087:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi087/log 2024-03-21T09:34:56.927 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T09:35:44.552 DEBUG:teuthology.misc:Transferring archived files from smithi100:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi100/log 2024-03-21T09:35:44.554 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T09:36:36.477 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi105/log 2024-03-21T09:36:36.479 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-21T09:37:56.573 INFO:tasks.cephadm:Removing cluster... 2024-03-21T09:37:56.573 DEBUG:teuthology.orchestra.run.smithi087:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --force 2024-03-21T09:37:56.764 INFO:teuthology.orchestra.run.smithi087.stdout:Deleting cluster with fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T09:37:57.343 DEBUG:teuthology.orchestra.run.smithi100:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --force 2024-03-21T09:37:57.653 INFO:teuthology.orchestra.run.smithi100.stdout:Deleting cluster with fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T09:37:58.258 DEBUG:teuthology.orchestra.run.smithi105:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 75a2f1f0-e75f-11ee-95cd-87774f69a715 --force 2024-03-21T09:37:58.653 INFO:teuthology.orchestra.run.smithi105.stdout:Deleting cluster with fsid: 75a2f1f0-e75f-11ee-95cd-87774f69a715 2024-03-21T09:37:59.406 INFO:tasks.cephadm:Removing cephadm ... 2024-03-21T09:37:59.406 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T09:37:59.488 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T09:37:59.507 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/cephadm 2024-03-21T09:37:59.522 INFO:tasks.cephadm:Teardown complete 2024-03-21T09:37:59.522 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-21T09:37:59.532 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-21T09:37:59.533 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T09:37:59.535 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T09:37:59.550 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-21T09:37:59.681 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-21T09:37:59.691 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-21T09:37:59.691 DEBUG:teuthology.orchestra.run.smithi087:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:37:59.694 DEBUG:teuthology.orchestra.run.smithi100:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:37:59.696 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-21T09:37:59.708 INFO:teuthology.orchestra.run.smithi087.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:37:59.710 INFO:teuthology.orchestra.run.smithi105.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:37:59.710 INFO:teuthology.orchestra.run.smithi100.stderr:bash: line 1: ntpq: command not found 2024-03-21T09:37:59.714 INFO:teuthology.orchestra.run.smithi087.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:37:59.714 INFO:teuthology.orchestra.run.smithi087.stdout:=============================================================================== 2024-03-21T09:37:59.714 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 55 -196us[ -196us] +/- 87ms 2024-03-21T09:37:59.714 INFO:teuthology.orchestra.run.smithi087.stdout:^* hv02.front.sepia.ceph.com 3 6 377 55 -245us[ -255us] +/- 57ms 2024-03-21T09:37:59.714 INFO:teuthology.orchestra.run.smithi087.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +480us[ +480us] +/- 77ms 2024-03-21T09:37:59.714 INFO:teuthology.orchestra.run.smithi087.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:37:59.795 INFO:teuthology.orchestra.run.smithi100.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:37:59.795 INFO:teuthology.orchestra.run.smithi100.stdout:=============================================================================== 2024-03-21T09:37:59.795 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 58 -191us[ -188us] +/- 87ms 2024-03-21T09:37:59.796 INFO:teuthology.orchestra.run.smithi100.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -232us[ -230us] +/- 57ms 2024-03-21T09:37:59.796 INFO:teuthology.orchestra.run.smithi100.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 +476us[ +476us] +/- 77ms 2024-03-21T09:37:59.796 INFO:teuthology.orchestra.run.smithi100.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:37:59.800 INFO:teuthology.orchestra.run.smithi105.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-03-21T09:37:59.800 INFO:teuthology.orchestra.run.smithi105.stdout:=============================================================================== 2024-03-21T09:37:59.800 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 54 -254us[ -254us] +/- 87ms 2024-03-21T09:37:59.800 INFO:teuthology.orchestra.run.smithi105.stdout:^* hv02.front.sepia.ceph.com 3 6 377 56 -263us[ -277us] +/- 57ms 2024-03-21T09:37:59.801 INFO:teuthology.orchestra.run.smithi105.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 50 +436us[ +436us] +/- 77ms 2024-03-21T09:37:59.801 INFO:teuthology.orchestra.run.smithi105.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-03-21T09:37:59.802 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-21T09:37:59.812 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-21T09:37:59.813 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-21T09:37:59.901 DEBUG:teuthology.orchestra.run.smithi087:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T09:38:00.064 DEBUG:teuthology.orchestra.run.smithi100:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T09:38:00.155 DEBUG:teuthology.orchestra.run.smithi105:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-03-21T09:38:00.250 DEBUG:teuthology.orchestra.run.smithi087:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:38:00.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:38:00.284 DEBUG:teuthology.orchestra.run.smithi100:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:38:00.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:38:00.317 DEBUG:teuthology.orchestra.run.smithi105:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-03-21T09:38:00.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:38:00.346 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-21T09:38:00.357 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-21T09:38:00.372 INFO:teuthology.task.internal:Duration was 3529.813524 seconds 2024-03-21T09:38:00.372 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-21T09:38:00.385 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-21T09:38:00.385 DEBUG:teuthology.orchestra.run.smithi087:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T09:38:00.387 DEBUG:teuthology.orchestra.run.smithi100:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T09:38:00.390 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-21T09:38:00.452 INFO:teuthology.orchestra.run.smithi105.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:38:00.462 INFO:teuthology.orchestra.run.smithi087.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:38:00.478 INFO:teuthology.orchestra.run.smithi100.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-03-21T09:38:00.836 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-21T09:38:00.836 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi087.front.sepia.ceph.com 2024-03-21T09:38:00.836 DEBUG:teuthology.orchestra.run.smithi087:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T09:38:00.859 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi100.front.sepia.ceph.com 2024-03-21T09:38:00.859 DEBUG:teuthology.orchestra.run.smithi100:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T09:38:00.883 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-03-21T09:38:00.883 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-21T09:38:00.903 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-21T09:38:00.904 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T09:38:00.907 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T09:38:00.926 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-21T09:38:00.981 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-21T09:38:00.981 DEBUG:teuthology.orchestra.run.smithi087:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T09:38:00.983 DEBUG:teuthology.orchestra.run.smithi100:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T09:38:01.006 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-21T09:38:01.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-21T09:38:01.395 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-21T09:38:01.395 DEBUG:teuthology.orchestra.run.smithi087:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T09:38:01.434 DEBUG:teuthology.orchestra.run.smithi100:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T09:38:01.472 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-21T09:38:01.516 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-21T09:38:01.533 DEBUG:teuthology.orchestra.run.smithi087:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:38:01.537 DEBUG:teuthology.orchestra.run.smithi100:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:38:01.541 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:38:01.569 INFO:teuthology.orchestra.run.smithi087.stdout:kernel.core_pattern = core 2024-03-21T09:38:01.596 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-03-21T09:38:01.631 INFO:teuthology.orchestra.run.smithi100.stdout:kernel.core_pattern = core 2024-03-21T09:38:01.650 DEBUG:teuthology.orchestra.run.smithi087:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:38:01.665 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:38:01.666 DEBUG:teuthology.orchestra.run.smithi100:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:38:01.681 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:38:01.681 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-21T09:38:01.697 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-21T09:38:01.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-21T09:38:01.718 INFO:teuthology.task.internal:Transferring archived files... 2024-03-21T09:38:01.721 DEBUG:teuthology.misc:Transferring archived files from smithi087:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi087 2024-03-21T09:38:01.721 DEBUG:teuthology.orchestra.run.smithi087:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T09:38:01.771 DEBUG:teuthology.misc:Transferring archived files from smithi100:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi100 2024-03-21T09:38:01.771 DEBUG:teuthology.orchestra.run.smithi100:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T09:38:01.813 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145/remote/smithi105 2024-03-21T09:38:01.813 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-21T09:38:01.852 INFO:teuthology.task.internal:Removing archive directory... 2024-03-21T09:38:01.852 DEBUG:teuthology.orchestra.run.smithi087:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T09:38:01.855 DEBUG:teuthology.orchestra.run.smithi100:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T09:38:01.857 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-21T09:38:01.900 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-21T09:38:01.913 INFO:teuthology.task.internal:Not uploading archives. 2024-03-21T09:38:01.913 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-21T09:38:01.925 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-21T09:38:01.925 DEBUG:teuthology.orchestra.run.smithi087:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T09:38:01.928 DEBUG:teuthology.orchestra.run.smithi100:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T09:38:01.930 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-21T09:38:01.940 INFO:teuthology.orchestra.run.smithi087.stdout: 262166 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 09:38 /home/ubuntu/cephtest 2024-03-21T09:38:01.942 INFO:teuthology.orchestra.run.smithi100.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 09:38 /home/ubuntu/cephtest 2024-03-21T09:38:01.955 INFO:teuthology.orchestra.run.smithi105.stdout: 262162 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 21 09:38 /home/ubuntu/cephtest 2024-03-21T09:38:01.957 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-21T09:38:01.968 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-21T09:38:01.991 INFO:teuthology.nuke:Checking targets against current locks 2024-03-21T09:38:02.017 DEBUG:teuthology.nuke:shortname: smithi087 2024-03-21T09:38:02.017 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:38:02.041 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi087.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:30:32.639937', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMfQ5mJ3bWnXAMBSktocr6hXpZC+vE9PwUKFZLqI7jI+ew4wzW7MsSXLDX2ELpAhsDehpUg+emWycV7YwX/Zhko='} 2024-03-21T09:38:02.046 DEBUG:teuthology.nuke:shortname: smithi100 2024-03-21T09:38:02.046 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:38:02.071 DEBUG:teuthology.nuke:shortname: smithi105 2024-03-21T09:38:02.071 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-21T09:38:02.076 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi100.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:30:32.635996', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAVVoaOBkE3CItZzR+zdxw5fQRlolyR8GPnQhe1YvWnJGeWD09s1hxpKJTGnOuvZmQDkItj4YHrSLawE8hZrMVE='} 2024-03-21T09:38:02.101 INFO:teuthology.orchestra.console.smithi087:Power off 2024-03-21T09:38:02.101 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T09:38:02.112 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-03-20_11:21:18-fs-wip-vshankar-testing-20240320.053002-testing-default-smithi/7612145', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-21 08:30:32.634397', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK4zlef7E7np1ld4jsMgNcjLRPZ+Q3Tck4iGeYrvwXb0VawcGgLDDTeIUkYSQnqH27s6fqHa2Ijd5Utipw+2dZA='} 2024-03-21T09:38:02.127 DEBUG:teuthology.orchestra.console.smithi087:power off output: Chassis Power Control: Down/Off 2024-03-21T09:38:02.127 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:02.145 INFO:teuthology.orchestra.console.smithi100:Power off 2024-03-21T09:38:02.145 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T09:38:02.159 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-03-21T09:38:02.168 DEBUG:teuthology.orchestra.console.smithi100:power off output: Chassis Power Control: Down/Off 2024-03-21T09:38:02.168 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:02.184 INFO:teuthology.orchestra.console.smithi105:Power off 2024-03-21T09:38:02.184 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-21T09:38:02.193 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T09:38:02.207 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-03-21T09:38:02.207 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:02.231 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-21T09:38:06.160 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:06.194 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:06.232 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:06.270 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is on 2024-03-21T09:38:06.304 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T09:38:06.342 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-21T09:38:10.274 DEBUG:teuthology.orchestra.console.smithi087:pexpect command: ipmitool -H smithi087.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:10.305 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:10.343 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:10.385 DEBUG:teuthology.orchestra.console.smithi087:check power output: Chassis Power is off 2024-03-21T09:38:10.414 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is on 2024-03-21T09:38:10.452 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-03-21T09:38:10.486 INFO:teuthology.orchestra.console.smithi087:Power off completed 2024-03-21T09:38:14.418 DEBUG:teuthology.orchestra.console.smithi100:pexpect command: ipmitool -H smithi100.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:14.453 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-21T09:38:14.528 DEBUG:teuthology.orchestra.console.smithi100:check power output: Chassis Power is off 2024-03-21T09:38:14.563 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-03-21T09:38:14.630 INFO:teuthology.orchestra.console.smithi100:Power off completed 2024-03-21T09:38:14.664 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-03-21T09:38:14.770 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/no 4-flush/no 5-workunit/suites/iogen}} duration: 3529.8135240077972 failure_reason: '"2024-03-21T08:48:58.152621+0000 mon.a (mon.0) 188 : cluster [WRN] Health detail: HEALTH_WARN 1/3 mons down, quorum a,c" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2024-03-21T09:38:14.770 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-21T09:38:14.844 INFO:teuthology.run:FAIL